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Abstract 

The  Royal  Australian  Air  Force  (RAAF)  needed  to  develop 
performance  measurements  (PMs)  to  support  its  Program  Management  and 
Budgeting  (PMB)  System.  This  research  assessed  the  use  of  the  concepts 
of  the  Theory  of  Constraints  (TOC)  to  develop  those  PMs.  The  literature 
indicated  that:  traditional  accounting  PMs  were  not  always  suitable  in 
an  environment  of  continuous  improvement;  Activity  Based  Costing  did  not 
provide  the  required  measures;  and  non-financial  PMs  were  supplanting 
these  older  measures.  Further,  governments  faced  unique  problems  in 
developing  PMs,  particularly  in  defining  the  outcomes  to  be  measured. 
Using  the  Critical  Theory  and  Action  Research  methodologies,  and  a  case 
study  approach,  the  research  derived  guidelines  from  PMB  to  develop  the 
RAAF  PMs,  examined  the  possible  use  of  TOC  in  government  organizations, 
developed  an  example  of  PMs  which  could  be  used  by  the  RAAF,  and 
compared  the  developed  PMs  with  the  PMB  guidelines.  The  research  found 
that  difficulties  arose  when  operationally  defining  the  outputs  of  the 
RAAF,  requiring  the  use  of  proxy  measures.  However,  with  slight 
modifications,  the  TOC  concepts  could  be  applied  to  the  development  of 
RAAF  PMs.  Development  would  be  an  iterative  process,  requiring 
refinements  as  users  gained  experience  with  the  PMs. 
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THE  DEVELOFMEMT  OF  A  PEHFOBKMICE  MEASUHEMEMT  CONCEPT 
FOR  THE  ROYAL  AUSTRALIAN  AIR  FORCE 


I .  Introduction 


Ovorviow 

This  introductory  chapter  provides  the  background  to  the  Royal 
Australian  Air  Force  (RAAF)  requirement  to  develop  metrics  to  support 
the  RAAF's  budgeting,  cost  management  and  performance  measurement 
requirements  both  in  the  1990s,  and  into  the  Twenty-First  Century. 

Based  on  this  background,  the  chapter  develops  a  research  objective 
aimed  at  providing  a  possible  performance  measurement  approach  to  meet 
the  RAAF  requirement. 

The  research  problem  is  stated  and  explained,  and  more  specific 
investigative  questions  are  described.  The  chapter  concludes  with  a 
discussion  on  the  potential  benefits  and  the  limitations  of  this 
research. 

Background 

The  Australian  Government  has  initiated  a  government  wide  program 
of  financial  management  reform.  In  response  to  this  reform,  the 
Australian  Department  of  Defence  (DOO)  has  implemented  a  Defence 
Financial  Management  Improvement  Program,  part  of  which  include^  the 
adoption  of  the  Government  mandated  Program  Management  and  Budgeting 
System  (PMB).  These  initiatives  were  directed  at  obtaining: 

a.  more  efficient  resource  allocation  through  better  budget 
decision-making  processes, 

b.  improved  effectiveness  by  focusing  on  results  through 
performance  measurement,  and 

c.  greater  efficiency  in  resource  use  by  improved  management  of 
programs  (3:2-4). 

The  DOD  implemented  PMB  in  1989;  however,  at  the  time  of 
conducting  this  research,  PMB  had  only  been  applied  as  a  high-level 
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resource  allocation  tool.  While  the  DOD  intends  to  apply  the  PMB 
process  throughout  all  levels  of  the  Department,  definitive  plans  for 
implementing  PMB  at  Base  or  Unit  level  are  yet  to  be  promulgated. 

PMB  provides  the  heart  of  the  accounting  requirements  for  the  DOD. 
The  Australian  Department  of  Finance  describes  PMB  as  being  a  goal 
oriented  program  designed  to  identify  objectives  and  outcomes,  and 
provides  a  structured  basis  to  plan,  budget,  implement,  r onitor,  and 
evaluate  a  government  department's  operations.  In  the  DOD,  PMB  is 
intended  to  provide  the  foundation  for  determining  the  cost- 
effectiveness  and  appropriateness  of  the  DOD  activities  designed  to 
provide  for  the  defence  of  Australia  (5:1-4). 

One  thrust  of  the  full  PMB  system  is  to  provide  cost  consciousness 
and  cost  visibility  at  all  levels  of  management.  Actions  required  to 
achieve  this  consciousness  and  visibility  in  the  logistics  support  arena 
include: 

a.  better  quantification  of  the  links  between  logistics 
resources  and  logistics  performance,  and  better  expression 
of  logistics  performance  in  terms  of  operational  value; 

b.  development  of  performance  indicators  at  all  levels  of 
logistics  management,  which  focus  attention  at  the  goal  of 
providing  effective  and  efficient  support  for  operations; 
and 

c.  development  of  supporting  management  information  and 
decision  support  systems  to  enable  detailed  activity  costing 
and  performance  assessment  at  all  levels.  (3:2-4, 5) 

On  9  May  90,  Assistant  Chief  of  the  Defence  Force  for  Logistics 
(ACLOG),  Maj  Gen  J.  C.  Grey,  presented  the  Defence  Logistics  Strategic 
Planning  Guide  (DLSPG)  to  the  Augmented  Chiefs  of  Staff  Committee. 

ACLOG  embraced  the  spirit  of  the  Government  directives,  and  embodied  the 
DOD  accounting  requirements  within  the  DLSPG  strategic  planning 
objectives. 

The  DLSPG  details  seven  strategic  planning  objectives  for 
logistics  development  to  be  addressed  by  the  Department.  Each  of  these 
strategic  objectives  is  supported  by  enabling  objectives,  and  by  broad 
strategies  to  achieve  the  objectives.  Listed  at  Appendix  A  are  the 
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DLSPG  cost  management  related  objectives  and  strategies  which  provided 
the  motivation  for  this  research. 

Essentially,  the  DLSPG  objectives  and  strategies  at  Appendix  A 
address  the  need  for: 

a.  the  development  of  methodologies  and  models  which  allow  effective 
budgeting,  resource  allocation  and  risk  analysis  to  support 
logistics  decision  making  processes; 

b.  the  establishment  of  performance  assessment  processes  and 
information  systems  that  focus  on  weapon  system  availability  and 
resource  allocations  against  weapon  system  priorities;  and 

c.  the  establishment  of  an  information  architecture  which  enables 
logistics  resource  entitlements,  availability  and  consumption  to 
be  linked  to  force  elements,  weapon  systems,  equipments  and 
logistics  activities  (3:B-2,D-l,E-2 ) . 

A  further  consideration  in  developing  performance  measures  is  the 
RAAF's  commitment  to  quality  and  continuous  improvement  through  the  RAAF 
Quality  (RAAFQ)  program  —  the  RAAF’s  implementation  of  total  quality 
management.  Metrics  that  allow  the  RAAF  to  track  progress  in  achieving 
RAAFQ  goals  are  an  important  part  of  the  RAAFQ  implementation  and 
management  processes. 

The  Problem 

One  of  the  underlying  themes  for  the  DLSPG  strategies  is  the 
requirement  for  a  conceptual  framework  which  can  be  used  to  develop 
performance  measurement  tools.  At  present,  such  a  conceptual  framework 
does  not  exist  within  the  Australian  Defence  Force  (ADF)  at  the  middle 
and  lower  levels  of  management. 

Further,  as  depicted  in  the  literature  review  in  Chapter  II, 
recent  literature  on  the  subject  of  performance  measurement  indicates 
that  the  traditional  commercial  performance  measures  based  on  the 
financial  data  of  an  organization  have  been  unsatisfactory  as  a  measure 
of  day-to-day  performance  of  the  organization.  In  addition,  the 
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development  of  performance  measures  for  not-for-profit  and  government 
organizations  presents  its  own  problems  which  require  the  development  of 
performance  measures  which  are  unique  to  each  not-for-profit  or 
government  organization. 

The  combination  of  the  search  for  new  performance  methodologies 
and  the  need  to  accommodate  the  unique  problems  of  the  government 
organization,  has  led  the  researcher  to  believe  that  there  are  no  "off- 
the-shelf"  performance  measures  which  the  RAAF  can  adopt.  Therefore, 
the  RAAF  needs  to  develop  and  adopt  its  own  performance  measures  to 
fulfil  the  Air  Force's  management  requirements  within  the  overall  thrust 
of  the  Department  of  Defence  strategic  objectives. 

One  approach  which  could  be  used  to  produce  a  conceptual  basis  for 
these  new  performance  measures  is  that  provided  in  E.  Goldratt's  Theory 
of  Constraints. 

Research  Objective 

The  purpose  of  this  research  was  to  assess  the  applicability  of 
using  the  performance  measurement  concepts  suggested  in  the  Theory  of 
Constraints  to  develop  performance  measures  to  monitor  the  achievement 
of  the  goals  and  objectives  of  the  RAAF. 

J^proach  Taken  in  Achieving  Research  Objective 

The  approach  taken  in  this  research  combines  "critical  theory"  and 
"action  research  "  methodologies,  using  a  case  study  as  the  vehicle  for 
presenting  the  research  analysis.  In  essence,  the  critical  theory 
approach  is  to  conduct  research  which  is,  at  the  outset,  designed  to 
facilitate  change.  The  approach  involves  the  use  of  both  concrete  and 
abstract  observations,  is  self-reflective  in  nature,  and  is  recognized 
as  being  potentially  value-laden.  Thus,  critical  theory  permits  the 
researcher  to  take  an  activist  role  in  the  research,  and  the  bias 
inherent  in  such  an  activist  role  is  acknowledged  at  the  start  of  the 
research  (34:328,333).  The  "action  research"  approach  is:  problem 
centered;  "bridge(8)  the  gulf  between  theory  and  practice";  provides  a 


means  of  further  action,  by  involving  the  members  of  the  organization 
being  researched  in  the  discussion  of  the  research;  and  emphasizes  the 
actual  use  and  dissemination  of  the  research  product  (33:186). 

Using  these  approaches  allowed  the  researcher  to  reach  conclusions 
and  recommendations  which  provided  a  set  of  pragmatic  measures  to  be 
used  as  the  basis  further  discussion  with  RAAF  policy  makers. 

Investigative  Questions 

There  are  four  investigative  questions  which  need  to  be  addressed 
to  achieve  the  purpose  of  the  research: 

a.  What  types  of  performance  measurement  indicators  are  required  in 
the  PMB  system  adopted  by  the  Australian  DOD,  and  what  attributes 
do  these  measures  require? 

b.  How  can  the  performance  measurement  concepts  of  the  TOC  be  adapted 
to  a  government  organization? 

c.  What  is  an  example  of  the  TOC  methodology  as  applied  to  a  RAAF 
Force  Element  Group  (FEG)? 

d.  Does  the  FEG  example  satisfy  the  performance  measurement 
requirements  of  the  PMB  process? 

Limitations 

There  are  four  limitations  placed  on  the  scope  of  the  research. 
First,  for  the  purposes  of  exploration  of  the  TOC  performance  measures 
in  the  RAAF  environment,  the  research  is  limited  by  taking  only  one 
Force  Element  Group,  the  Strike/Reconnaissance  Group,  as  the  basis  for  a 
case  study.  A  Force  Element  Group  (FEG)  is  a  combat  program  under  PMB, 
and  under  PMB,  FEGs  are  considered  to  be  the  building  blocks  for 
corporate  planning  for  results,  as: 

a.  they  are  generally  natural  groupings  of  elements  of  the  Defence 
Force  for  reporting  purposes, 

b.  they  can  be  readily  related  to  the  1987  'White  Paper'  and 
subsequent  policy  initiatives,  and 


c.  they  are  the  basis  for  outcome  reporting  under  CDF's  classified 
Operational  Readiness  Directive  (4:7). 

Second,  the  researcn  was  confined  to  a  review  of  the  literature 
and  an  exploratory  excunination  of  one  performance  measurement  concept. 
The  researcher  acknowledged  that  there  are  be  other  conceptual 
frameworks  which  could  be  used  by  the  RAAF,  for  example,  the  Computer- 
Aided  Manufacturing  -  International  Project,  or  the  Government 
Accounting  Standards  Board  "Service  Effects  and  Accomplishments  Project" 
(7;15). 

Third,  the  research  was  developed  from  a  theoretical  basis, 
adapting  new  approaches  and  philosophies  to  the  RAAF  environment.  This 
approach  was  taken  with  the  goal  that  the  next  stage  of  the  measurement 
concept  development  would  be  the  discussion  of  the  concepts  with  the 
potential  users  of  the  metrics. 

Finally,  the  surveying  of  potential  managers  to  ascertain  their 
cost  management  and  performance  measurement  requirements  was  not 
considered  feasible  due  to  perceived  problems  associated  with  the 
potential  managers'  lack  of  knowledge  of  management  accounting  and 
performance  measurement  techniques.  As  the  RAAF  currently  has  no  cost 
management  and  performance  measurement  system  at  the  middle  and  lower 
management  levels,  the  potential  users  of  a  new  cost  management  and 
performance  measurement  system  have  had  no  exposure  to  the  requirements 
of  accountability  for  resource  management.  An  education  program  will  be 
required  to  acquaint  these  levels  of  management  with  the  capabilities 
that  a  cost  management  and  performance  measurement  system  can  provide. 

Benefits  of  the  Research 

The  major  benefits  of  the  research  were: 
a.  the  provision  of  insight  into  the  difficult  problem  of  formulating 
performance  measurements  in  the  RAAF; 
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b.  the  provision  of  a  baseline  for  discussion  on  the  potential  use  of 
Theory  of  Constraint  performance  measurement  techniques  within  the 
RAAF;  and 

c.  the  provision  of  a  baseline  against  which  other  conceptual 
frameworks  for  developing  performance  measures  can  be  compared. 

Summary 

Chapter  I  describes  the  cost  management  reforms  which  the 
Australian  Government  has  initiated  in  the  Australian  Department  of 
Defence.  These  reforms  have  created  a  need  for  the  Royal  Australian  Air 
Force  to  develop  a  performance  measurement  system  to  meet  the  Air 
Force's  obligations  under  those  Defence  cost  management  reforms.  The 
need  has  prompted  this  research  to  identify  and  outline  one  possible 
performance  measurement  methodology,  using  contemporary  thought,  for  use 
within  the  RAAF. 

To  facilitate  the  required  research,  four  investigative  questions 
were  formulated  to  obtain  answers  to:  what  type  of  performance  measures 
are  required  by  the  PMB  process;  how  can  the  performance  measurement 
concepts  of  the  Theory  of  Constraints  be  adapted  to  a  RAAF  FEG;  what  is 
an  example  of  the  Theory  of  constraints  methodology  as  applied  to  a  RAAF 
FEG;  and  does  this  exeunple  satisfy  the  performance  measurement 
requirements  of  PMB? 

The  approach  taken  in  the  research  had  some  limitations  in  its 
scope,  these  limitations  being  dictated  by  the  time  and  distance 
constraints  placed  on  the  researcher.  The  scope  of  the  research  was 
limited  to  one  RAAF  FEG,  and  to  the  performance  measurement  concepts 
found  in  the  Theory  of  Constraints. 

The  research  provided  three  benefits:  the  development  of  insight 
into  the  difficult  problem  of  defining  performance  measures  for 
government  organizations,  the  provision  of  a  baseline  for  discussion  of 
the  concepts  of  the  Theory  of  Constraints  as  applied  to  government 


7 


organizations,  and  the  provision  of  a  baseline  against  which  other 
performance  measurement  concepts  for  the  RAAF  can  be  compared. 

Thesis  Organization 

Chapter  I  provides  an  introduction  to  the  research.  Chapter  II 
provides  a  review  of  the  current  literature  on  management  accounting  for 
performance  measurement,  specifically  referring  to  the  problems  of  using 
traditional  management  accounting  methodologies  as  a  basis  for 
formulating  performance  measures. 

Chapter  III  explains  the  methodology  used  to  conduct  the  research. 
The  methodology  includes  reviewing. the  requirements  of  PMB,  adapting  the 
concepts  of  The  Theory  of  Constraints  to  a  government  organization, 
developing  a  set  of  performance  measures  for  the  one  FEG,  and 
determining  whether  the  developed  measures  meet  the  PMB  guidelines  for  a 
performance  measure.  Chapter  IV  answers  the  research  questions,  and  the 
research  conclusions  and  recommendations  made  as  a  result  of  the  study 
are  contained  in  Chapter  V. 


ZI .  Litaratur*  naviaw 


Ovaxvlaw 

There  were  four  main  objectives  for  conducting  this  literature 
review.  The  first  objective  was  to  examine  the  reasons  why  many  authors 
believe  that  traditional  management  accounting  approaches  are 
inappropriate  for  the  formulation  of  performance  measurements  for 
contemporary  organizations.  The  second  objective  was  to  examine  why 
they  believe  the  strong  recent  trend  to  Activity  Based  Costing  did  not 
fully  overcome  the  problems  of  the  traditional  approach.  The  third 
objective  was  to  investigate  the  growing  preference  for  non-f inancial 
performance  measures.  Finally,  the  fourth  objective  was  to  identify  the 
unique  problems  faced  by  not-for-profit  and  government  organizations  in 
formulating  performance  measurements. 

The  Inadequacy  of  Traditional  Accounting  Measures 

Traditional  cost  management  and  performance  measures  had  their 
origins  in  the  scientific  management  movements  of  80-100  years  ago.  The 
cost  accounting  model  which  evolved  from  these  origins  was  based  on  the 
use  of  standard  costs,  and  the  periodic  analysis  of  variances  of  actual 
costs  from  these  standard  costs.  The  model  was  predicated  on  an 
environment  which  featured  the  mass  production  of  standardized  products, 
where  efficiency  could  be  achieved  by  maximizing  the  output  produced  by 
direct  labor  and  the  machines  operated  by  that  labor.  As  a  consequence, 
the  accounting  systems  which  emerged  required  the  development  of 
standards  for,  and  the  subsequent  tracking  of,  efficiencies  based  on  the 
individual  worker  and  individual  machines  (21:16). 

During  the  19808,  in  an  environment  where  organizations  war» 
attempting  to  achieve  the  goals  of  total  quality  control,  just-in-time 
production,  the  rapid  introduction  of  new  products  produced  in  low 
volumes,  and  the  concepts  of  ongoing,  continuous  improvement  activities, 
the  use  of  traditional  management  accounting  approaches  to  performance 
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measurement  of  organizations  came  under  increasing  attack.  According  to 
Kaplin,  the  traditional  cost  model  was  accused  of  being  a  major 
stumbling  block  to  many  organizations*  efforts  to  become  high  quality, 
responsive,  and  flexible  organizations. 

In  many  cases,  despite  improvements  provided  in  the  new  operating 
environment,  such  as  lower  defect  rates  and  reduced  throughput  times, 
unit  costs  and  financial  efficiency  measures  had  not  shown  any 
improvement.  Examples  were  provided  of  cases  where  these  measures  had 
shown  that,  contrary  to  reality,  efficiency  had  even  deteriorated.  In 
other  cases,  the  traditional  financial  performance  measures  actually 
inhibited  quality  and  process  improvement  activities  (21:2,16-17). 

A  seminal  work  in  the  exploration  of  the  decline  in  relevance  of 
the  traditional  cost  model  was  provided  by  Johnson  and  Kaplin  in  their 
book  "Relevance  Lost:  The  Rise  and  Fall  of  Management  Accounting"  (19). 
The  book  described  in  detail  the  inadequacy  of  the  traditional  methods, 
and  their  findings,  along  with  some  contributions  from  other  academics, 
practicing  cost  management  and  production  management  professionals,  are 
discussed  in  the  following  paragraphs.  The  generalizations  in  these 
paragraphs  were  provided  by  the  respective  authors,  based  on  their 
research  and  investigations. 

Misleading  Target  for  Managerial  Attention.  The  traditional  cost 
model  failed  to  provide  relevant  measures  that  reflect  the  technology, 
products  and  processes  in  which  the  contemporary  organization  operated. 
Financial  managers  who  relied  exclusively  on  periodic  financial 
statements  for  their  view  of  the  organization  became  isolated  from  the 
real  value  creating  operations  of  the  organization.  Consequently,  they 
failed  to  recognize  when  the  accounting  numbers  were  no  longer  providing 
relevant  or  appropriate  measures  of  the  organizations  operations 
(20:3-4) . 

Inappropriate  Focus.  The  old  system  was  initially  adapted  to 
value  both  units  of  unsold  product  at  the  end  of  the  period  (finished 
and  in-process  inventories)  and  units  of  production  sold  during  the 
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period.  Inventory  costing  therefore  provided  inventory  values  to  report 
on  the  balance  sheet,  and  manufacturing  expenses  to  match  against 
revenues  on  the  income  statement.  These  costings  did  not  reflect  the 
true  performance  of  the  organization  (20:130). 

Adverse  Consequences  of  Direct  Labor  Allocation  Systems.  The 
major  distortion  in  the  traditional  cost  model  was  caused  by  the  use  of 
direct  labor  to  allocate  overhead  costs  to  products  and  services.  This 
methodology  prompted  cost  center  managers  and  product  managers  to  focus 
their  cost-reduction  attention  solely  at  direct  labor  savings.  In  the 
contemporary  environment,  with  overhead  burden  rates  of  400  to  1000 
percent  not  unusual,  small  savings  in  direct  labor  time  had  large 
impacts  on  cost  distributions  and  product  costs  (20:188). 

The  focus  on  labor  could  result  in  enormous  amounts  of  time  and 
effort  being  committed  to  the  detailed  recording  and  processing  of  labor 
time,  and  it  was  not  unusual  to  see  thousands  of  dollars  of  management 
time  being  devoted  to  saving  tenths  of  hours  of  direct  labor  time. 

Then,  at  the  end  of  an  a  hunting  period,  further  management  time  was 
spent  analyzing  unfavorable  labor  variances  that  seemed  to  involve 
trivial  amounts  of  actual  labor  hour  variation. 

Such  concentration  on  labor  hours  distracted  attention  from  the 
cost  of  overheads,  the  area  where  costs  were  actually  increasing  most 
rapidly.  The  problem  of  allocating  overhead  using  labor  hours  was 
further  compounded  by  the  fact  that,  if  a  process  manager  succeeds  in 
reducing  growth  in  an  overhead  cost  category,  the  benefit  is  distributed 
broadly  to  all  cost  centers  and  processes  in  the  organization.  This 
created  a  behavioral  environment  where  managers,  to  achieve  the  maximum 
reductions  in  the  allocated  costs  for  their  individual  processes,  shied 
away  from  reducing  overheads,  and  concentrated  on  reducing  their  direct 
labor  charges,  since  that  was  the  cost  driver  by  which  all  other  costs 
were  attached  to  their  cost  centers  and  processes  (20:188). 

Based  on  this  view  of  management,  any  process  that  required 
relatively  large  amounts  of  direct  labor  seemed  to  be  very  expensive. 
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In  many  cases  it  was  "cheaper"  to  find  a  supplier  that  could  produce  the 
labor-intense  component,  subassembly  or  service  than  for  the  cost  center 
to  produce  it.  Thus,  subcontracting  apparently  lowered  costs,  prompting 
the  organization  to  buy  rather  than  produce  (20:189). 

Unfortunately,  in  many  cases,  overhead  costs  tended  to  rise  with 
increased  eunounts  of  subcontracting.  Subcontracting  imposed  additional 
demands  on  the  purchasing  department  to  generate  specifications  for  the 
component  and  to  investigate  quality  vendors;  on  the  scheduling 
department  to  provide  delivery  schedules  to  the  vendor;  on  the  receiving 
and  inspection  department  to  process  incoming  items;  on  materials 
handling  departments  to  place  purchased  components  into  storage  and 
bring  them  out  to  production  when  needed;  and  on  the  accounts  payable 
department  to  pay  the  vendor.  All  of  the  new  support  activities  added 
to  overhead  costs.  But  the  newly  added  costs  were  not  traced  to  the 
purchased  component  because  it  had  zero  direct  labor  content.  Instead, 
the  higher  overhead  costs  were  shifted  to  the  labor-intense  products  and 
processes  still  remaining  in  the  plant,  making  the  remaining  products 
and  services  "more  expensive",  prompting  further  subcontracting. 

Often,  the  direct  labor  allocation  base  also  distorted  product  costs 
and  introduced  unintended  cross  subsidies  by  shifting  costs  from  less 
labor-intense  products  to  more  labor-intense  products.  Even  when  cost 
centers  used  a  flexible  budget  which  identified  and  separated  variable 
and  fixed  expense,  the  variable  portion  of  costs  was  assumed  to  vary 
with  direct  labor  activity.  In  addition  to  introducing  unintended  cross 
subsidies  among  products,  the  direct  labor  overhead  allocation  system 
inhibited  cost  planning  and  cost  control  (20:188-190). 

Lack  of  Tiawllness.  Typical  1980s  cost  accounting  systems  failed 
to  provide  information  in  a  timely  manner.  As  a  result  of  the  various 
time-consuming  operations  required  to  prepare  periodic  costing 
statements,  it  was  seldom  possible  to  obtain  cost  statements  of  the 
conventional  type  very  soon  after  the  period  has  closed.  Typically,  an 
accounting  report  for  a  period  would  appear  just  before  the  middle  of 


the  subsequent  period,  e.g.  monthly  reports  would  appear  in  the  middle 
of  the  following  month.  Despite  a  great  deal  of  attention  being  devoted 
to  performing  "fast  closes",  even  with  the  fastest  of  closings,  the  cost 
information  was  produced  too  late  to  help  short-term  process  control. 
Unfortunately,  if  a  problem  arises,  process  managers  need  to  deal  with 
it  immediately;  they  cannot  wait  until  sometime  the  following  month  to 
discover  the  process  variances.  Therefore,  Johnson  and  Kaplin  stated 
that  it  may  be  more  important  to  make  available  essential  interim 
control  information  for  managerial  purposes  than  to  prepare  overall 
statements.  Incomplete  details  of  an  operation,  obtained  promptly,  may 
be  of  more  use  than  complete  information  made  available  only  after  a 
considerable  lapse  time  (20:193). 

Information  Produced  at  Too  Aggregate  a  Level.  Apart  from 
arriving  too  late  to  be  of  much  use  for  controlling  the  production 
system,  the  cost  management  information  was  usually  produced  at  too 
aggregate  a  level  to  be  able  to  pinpoint  the  source  of  adverse,  or  even 
favorable,  production  variances  (20:194).  In  addition,  the  information 
was  too  technical,  and  was  not  easily  understood  by  non-accounting 
managers  (11:33). 

Inconsistency  Between  Reports  and  Reality.  The  cost  data  derived 
using  the  traditional  costing  method  incorporated  allocations  that  were 
inconsistent  with  the  actual  factory  production  process  and  product  mix. 
Reports  were  usually  produced  on  a  cost  center  basis,  and  did  not 
monitor  adequately  the  costs  of  business  processes  which  spanned 
interrelated  departments  or  units.  In  addition,  the  reports  failed  to 
support  the  information  requirements  of  modern  techniques  such  as  Just- 
In-Time  and  Total  Quality  Management  (11:33;  20:194). 

Capacity  for  Manipulation  of  Results.  Problems  with  managing  for 
short-term  financial  objectives  arose  because  operating  managers  learned 
that  there  were  a  variety  of  ways  to  meet  profit  and  return  on 
investment  goals.  Managers  discovered  that  profits  could  be  "earned" 
not  just  by  selling  more  or  producing  for  less,  but  also  by  engaging  in 
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a  variety  of  nonproductive  activities,  such  as:  exploiting  accounting 
conventions,  engaging  in  financial  entrepreneurship,  and  temporarily 
reducing  discretionary  expenditures.  Historical  cost  accounting 
procedures  and  "generally  accepted  accounting  principles"  (GAAP)  provide 
ample  opportunities  for  executives  to  manage  their  income  and  investment 
measures.  As  one  example,  considerable  discretion  existed  for  the  timing 
of  revenue  and  expense  recognition  so  as  to  exhibit  steady  earnings 
growth  or  to  meet  budgeted  income  or  expense  goals  for  the  current 
period  (20:196). 

External  Reporting  Bias.  Traditional  cost  accounting  systems 
attempted  to  satisfy  three  goals:  to  allocate  certain  period  costs  to 
products  so  that  financial  statements  can  be  prepared  monthly, 
quarterly,  and  annually  (external  focus);  to  provide  process  control 
infoirmation  to  cost  center  managers  (internal  focus);  and  to  provide 
product  cost  estimates  to  product  and  business  managers  (internal 
focus).  Typically,  only  a  single  cost  system  was  used  for  these  three 
quite  different  goals.  Because  financial  accounting  considerations  had 
been  dominant,  only  the  first  of  the  three  goals  was  accomplished  well 
(20:231) . 

Other  comments  on  the  external  reporting  bias  included: 

a.  Traditional  cost  systems  were  meant  primarily  to  value  inventory 
and  provide  data  for  profit  and  loss  statements  and  balance  Sheets 
(27:36) . 

b.  Reports  tended  to  be  pre-occupied  with  bottom-line  profits  and  did 
not  focus  on  the  critical  success  factors,  such  as  increasing 
efficiency,  better  quality,  improving  customer  service  and  product 
flexibility  (11:33). 

There  were  numerous  other  articles  which  directed  criticism  at  the 
traditional  cost  model.  The  thrust  of  their  criticisms  echo  the 
comments  of  Johnson  and  Kaplin  (10:38;  11:31-33;  13:56;  22:36;  23:11; 
25:42;  28:28;  32:20). 
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Conclusions  in  th«  Litsrstur*.  The  literature  reviewed  above 
drew  a  number  of  conclusions  regarding  traditional  performance  measures. 
First,  Kaplin  stated  that  the  traditional  summary  measures  of  local 
performance  —  such  as  direct  labor  and  machine  efficiencies,  and 
absorption  ratios  --  "are  harmful  and  probably  should  be  eliminated." 
Instead  direct  measurement  was  needed  of  quality,  process  times, 
delivery  performance,  and  any  other  operating  performance  criteria  that 
organizations  may  want  to  improve. 

Kaplin  went  on  to  say  that  actual  resource  consumption  and 
aggregated  cost  figures  should  be  compared  not  to  cost  standards,  but  to 
trends  of  past  actuals.  The  standards  for  today's  organizations  should 
require  improvement  from  the  levels  established  by  outputs  from  activity 
in  previous  operating  periods.  Results  should  be  presented  graphically 
in  the  form  of  trends  of  output  and  resource  consumption,  with  these 
simpler  measures  replacing  "the  myriad  of  numbers  currently  reported  in 
aggregate  monthly  financial  summaries". 

In  contrast  to  current  usage,  financial  summaries  should  be 
informational,  and  not  control,  measures.  The  summaries  would  be 
attention*-getting  and  aggregate  score-keeping  in  nature,  and  perhaps 
computed  only  semi-annually  or  annually,  to  provide  a  comprehensive, 
interorganizational  measure  of  financial  activity  for  key  activities 
(21:35-36) . 

As  for  the  design  of  new  performance  measures,  Kaplin  states: 

Just  how  to  design  a  hierarchical,  comprehensive  system  of  local 
shop-floor  measures,  departmental  and  plant  measures  and 
divisional  performance  measures  remains  a  task  for  future  research 
and  experimentation  (21:10). 

The  conclusion  from  reviewing  this  literature  is  that  traditional 
cost  measures  may  not  be  the  correct  measures  for  adoption  by  the  RAAF. 

A  Shift  to  Activity  Based  Costing 

Many  of  the  articles  that  criticized  the  traditional  cost  model 
offer  Activity  Based  Costing  (ABC)  as  the  preferred  alternative  method 
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to  the  old  model  (11;  13;  22;  23).  Cooper  describes  the  ABC  system  as 
follows: 

Activity-based  cost  systems  are  distinguished  from  their 
conventional  counterparts  by  the  cost  drivers,  or  allocation 
bases,  they  use  to  relate  the  consumption  of  inputs  to  products. 
Conventional  systems  rely  solely  on  unit-level  cost  drivers,  such 
as  direct  labor  hours  or  dollars,  machine  hours,  and  materials 
dollars.  A  unit-level  cost  driver  assumes  that  inputs  are 
consumed  in  direct  proportion  to  the  number  of  units  produced. 
Activity-based  systems,  while  retaining  unit-level  drivers,  also 
use  nonunit-level  ones.  Nonunit-level  drivers  assume  that  costs 
are  not  consumed  in  direct  proportion  to  the  number  of  units 
produced.  These  drivers  include  the  number  of  part  numbers, 
number  of  vendors,  and  number  of  setups.  (9:304) 

However,  the  shift  to  ABC  was  perhaps  not  the  answer  for  the 

operational  performance  measures  required  by  an  organization.  While 

Johnson  supported  the  move  to  ABC,  he  did  not  believe  that  ABC  in  itself 

was  a  tool  for  continuous  improvement.  In  a  recent  article  he  stated: 

The  belief  that  activity-based  cost  information  improves  a 
company's  performance  is  a  delusion.  No  accounting  information, 
not  even  activity-based  cost  management  information,  will  help 
companies  achieve  competitive  excellence.  As  a  tool  to  improve 
cost  accounting  information  ABC  is  impeccable,  beyond  that  it  is 
sna)ceoil  (19:13). 

Johnson  contended  that  all  management  control  information  comes 
from  customers  and  processes,  in  real  time,  and  the  aggregation  of  cost 
accounting  data  did  not  reflect  customer  expectations  cr  process 
capabilities.  As  such,  Johnson  advocated  that  more  attention  be  given 
to  Statistical  Process  Control  (SPC)  charts  maintained  by  both  workers 
and  managers  alike.  All  members  of  the  organization  should  understand 
how  to  translate  information  from  these  SPC  charts  into  actions  which 
result  in  continuous  improvement  of  the  organization  as  a  whole. 

To  be  able  to  respond  quickly  to  the  need  for  change  required  a 
flexibility  in  not  having  to  wait  for  accounting-based  instructions  from 
above.  Both  internal  and  external  customers  would  offer  new  types  of 
information  on  how  they  do  or  don't  like  what  they  get,  and  real-time 
instructions  would  come  from  processes,  customers,  and  from  problem 
solving  teams  (19:13). 

ABC  had  the  problem  that  it  was  not  real-time.  ABC  was  really 
only  reconstituted  accounting  information,  and  it  would  not  help  workers 
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maintain  competitive  processes,  or  help  satisfy  customers  quickly. 
According  to  Johnson,  "no  accounting  information  —  not  even  new  and 
improved  ABC  —  ever  indicated  if  a  customer  was  satisfied,  if  a  process 
were  in  control,  or  what  time  it  takes  to  accomplish  a  job." 

Further,  ABC  did  not  identify  the  changes  that  must  be  made  to 
become  a  truly  flexible  producer.  It  simply  identified  steps  that 
reduced  costs  or  raised  margins,  while  doing  business  as  usual. 
Activity-based  cost  information  did  nothing  to  change  old  style 
management  behavior. 

Johnson  concluded  by  recommending  organizations  to  invest  in 
customer  focused  problem-solving  initiatives  to  remove  constraints  that 
cause  variation,  delay,  and  excess  in  processes.  A  focus  on  the 
reduction  of  variation  and  leadtime,  he  believes,  will  lead  to  an  almost 
automatic  reduction  in  costs  (19:13-14). 

Johnson  was  not  the  only  critic  of  the  rush  to  ABC  for  the  purpose 
of  providing  performance  measures.  Piper  and  Walley  stated  that  ABC  was 
inappropriate  at  the  strategic  level,  and  that  at  the  operational  level, 
management  could  be  better  motivated  by  physical  measures,  e.g.  delivery 
speed,  delivery  reliability  etc,  without  a  dubious  cost  being  ascribed 
to  them,  or  if  a  cost  was  ascribed,  its  behavioral  and  operational  role 
was  clearly  delineated.  They  concluded  that  the  analysis  of  activity 
was  definitely  a  worthwhile  activity,  but  that  ABC  may  not  be 
(25:44,54) . 

The  general  thrust  of  the  Johnson  and  Piper  arguments  was  that 
neither  ABC,  nor  any  other  purely  accounting-based  set  of  measures,  will 
provide  the  types  of  performance  measures  required  in  the  contemporary 
environment  of  continuous  improvement.  These  arguments  were  in  accord 
with  another  recent  trend  towards  the  use  of  more  non-f inancial  based 
indicators  of  organizational  performance. 
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Incr«as«d  Us*  of  Non-Financial  Data 

The  retreat  from  using  purely  accounting  based  information  was 
supported  by  a  growing  number  of  academics  and  practicing  management 
advisors.  Johnson  and  Kaplin  contended  that  measuring  and  reporting  a 
variety  of  non-financial  indicators  was  more  important  than  measuring 
monthly  or  quarterly  profits.  Organizations  should  be  developing 
indicators  based  on  the  key  measures  of  performance  success,  with 
organizations  wanting  to  become  lower  cost  producers  developing 
productivity  measures  which  show  trends  in  their  ability  to  produce  more 
with  less  (20:256-7). 

Puxty  and  Lyall  argued  that  non-financial  data  had  a  greater 
impact  on  the  organization's  performance  than  financial  data  (26:46), 
while  Sheridan  observed  that  the  Japanese  approach  was  to  use  non- 
financial  indicators,  particularly  on  the  shop  floor,  and  that  UK 
companies  too  were  adopting  these  types  of  indicators.  The  reason  for 
this  trend  was  that  these  non-financial  indicators  did  have  costing 
implications,  and  that  an  improvement  in  these  indicators  nearly  always 
resulted  in  a  better  ability  of  the  organization  to  meet  cost  and  profit 
targets  (32:24). 

Providing  a  slightly  different  point  of  view  was  Ostrenga.  He 
argued  that  performance  measures  should  represent  a  mix  of  financial  and 
non-financial  measures,  with  a  pronounced  increase  in  the  use  of  non- 
financial  measures  at  the  middle  management  level.  Such  performance 
measures  assisted  in  the  continuous  improvement  philosophy  by  focusing 
on  the  significant  activity  levels,  and  measuring  the  drivers  of  those 
activities.  Further,  while  operational  in  nature,  these  measures  must 
still  be  aligned  with  the  organization's  goals  and  critical  success 
factors  to  ensure  that  management  behavior  was  pointed  in  the  right 
direction,  and  to  prevent  situations  where  the  manager  optimized  an 
individual  area,  department  or  function  at  the  expense  of  sub-optimizing 
the  whole  organization  (24:46). 
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Citing  his  experience  with  clients,  ostrenga  recommended 
developing  a  set  of  measures  which  provided  a  balance  between: 

a.  Effectiveness.  Was  the  organization  doing  the  right  thing?  This 
involved  a  comparison  of  planned  output  with  actual  output. 

b.  Efficiency.  How  well  was  the  organization  achieving  its  output? 
This  involved  comparisons  of  planned  input  with  actual  input. 

c.  Productivity.  How  much  output  was  the  organization  getting  for  a 
given  input? 

d.  Utilization.  How  was  the  organization  using  its  resources  such  as 
inventory,  asset  turnover,  etc? 

The  goal  of  this  set  of  measures  was  to  assist  management  by  tying 
controllable  cost  measures  to  a  more  comprehensive  view  of  managing  the 
organization's  resources  (24:47). 

Performance  Measurement  in  Oovemment  Organizations 

For  the  for-profit  organizations,  the  allocation  of  resources  to 
produce  output  was  guided  automatically  by  the  market  structure.  A  cost 
benefit  for  any  resource  allocation  alternative  was  provided  through  a 
market  determined  value  of  output,  and  this  output  value  could  be 
compared  to  the  cost  of  the  inputs  to  provide  that  output.  The  dollar 
value  of  sales  could  be  treated  as  output,  while  the  dollar  value  of  the 
cost  of  sales  could  be  treated  as  input.  This  allowed  the  use  of  the 
concept  of  profit  as  a  measure  of  both  efficiency  and  effectiveness  in 
the  allocation  of  resources. 

In  contrast,  for  most  government  organizations,  there  were  no 
monetary  based  markets  in  which  to  value  output  in  dollar  terms. 
Accordingly,  there  was  no  single  indication  of  performance  comparable  to 
profit  to  measure  the  "value"  of  services  provided,  and  managers  were 
driven  to  some  combination  of  judgement  and  input  cost  minimization  as  a 
means  for  searching  for  optimum  performance  (2:6;  8:50).  A  corollary  to 
these  points  was  that  for  many  of  the  most  important  decisions  in  a 
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government  organization,  there  was  no  accurate  way  of  estimating  the 
relationship  between  inputs  and  outputs  (2:40). 

As  Drebin  stated,  the  situation  is  further  complicated  when 
considering  the  products  of  governmental  agencies.  Many  of  the  products 
of  these  agencies  were  often  regarded  as  being  subjective  and  not  being 
susceptible  to  measurement.  However,  this  perception  of  subjectivity 
should  not  interfere  in  satisfying  the  urgent  need  to  develop  useful 
measures  for  evaluating  governmental  programs  and  services.  (12:3) 
Anthony  and  Herzlinger  concurred  that,  in  many  government  organizations, 
no  good  quantitative  measure  of  output  existed  (2:5). 

Unfortunately,  current  practice  had  a  tendency  to  focus  on  the 
amount  of  activity,  or  the  production  of  some  intermediate  product, 
rather  than  focusing  on  final  results.  These  activities  and 
intermediate  products  were  easy  to  measure,  but  the  use  of  the 
associated  measures  often  caused  managers  to  lose  sight  of  the  real 
purpose  for  the  activities  being  managed  (12:3). 

A  further  problem  with  measurement  in  government  organizations  was 
that  there  may  be  a  general  agreement  on  the  direction  a  progreun  was  to 
follow,  but  no  agreement  about  how  a  favorable  result  should  be  defined. 
Obviously,  how  success  was  defined  had  an  important  influence  on 
measuring  program  results  (12:7). 

In  many  cases,  performance  may  not  be  able  to  be  measured  in 
absolute  terms,  but  could  be  measured  in  relative  terms  which  permitted 
inter juristictional  and  interperiod  comparisons.  While  it  may  not  be 
possible  to  identify  outcomes  in  precise  values,  it  may  be  possible  to 
say  the  results  are  better  or  worse  than  the  results  of  a  previous 
period,  or  for  a  similar  program  in  another  jurisdiction  (12:7). 

One  method  for  achieving  these  relative  measures  was  through  the 
use  of  proxy  measures  which  could  provide  quantifiable  indications  of 
performance  when  direct  measurement  was  difficult  or  impossible.  A 
proxy  measure  was  correlated  with  the  desired  outcome,  but  was  more 
easily  measured.  Naturally,  the  credibility  of  any  proxy  depends  on  how 
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closely  correlated  the  proxy  is  to  actual  outcomes.  Orebin  offered  a 
warning  about  the  use  of  proxies,  stating  that  correlation  did  not 
necessarily  prove  a  cause  and  effect  relationship.  While  it  was 
reasonable  to  assume  that  employees  would  be  motivated  to  maximize 
performance  as  it  was  measured,  and  although  there  may  have  been  a 
correlation  between  results  and  the  proxy  in  the  past,  emphasizing  the 
proxy  could  cause  the  relationship  between  the  proxy  and  the  desired 
outcome  to  change  as  efforts  were  made  to  improve  measured  rather  than 
actual  performance  (12:4). 

Drebin  concluded  that,  given  the  need  for  developing  performance 
measures  that  reflected  the  actual  results  of  government  programs,  much 
work  remained  in  establishing  useful  measures,  and  there  was  likely  to 
be  considerable  controversy  surrounding  the  adoption  of  any  specific 
measure  (11:7). 

The  search  for  appropriate  measures  for  a  government  organization 
was  not  an  easy  one.  In  his  review  of  budgeting  developments  in  five 
industrialized  countries,  Schick  provided  examples  of  the  difficulties 
being  faced  by  governments  in  addressing  performance  measures. 

In  Canada,  the  Auditor  General  of  Canada  in  his  1987  annual  report 
stated:  "The  concept  is  simple  —  objectives,  results,  and  resources 
should  all  be  linked.  The  application  is  difficult.”  He  also  found 
that  "managers  have  considerable  difficulty  translating  (budget) 
objectives  into  clear,  measurable  and  attainable  statements  of  purpose" 
(31:28) . 

In  addition,  under  the  Increased  Ministerial  Authority  and 
Accountability  (IMAA)  reforms  being  undertaken  by  the  Canadian 
Government,  the  IMAA  guidelines  recognized  that  developing  precise 
quantitative  results  statements  was  not  always  feasible,  and  therefore, 
where  appropriate,  qualitative  or  proxy  measures  should  be  used  instead 
(31:28). 

In  relation  to  PMB  in  Australia,  Schick  stated  that  government 
departments  had  difficulty  in  defining  policy  aims  and  in  using 
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performance  indicators  to  measure  results.  An  official  Australian 
Government  report  found  that  progress  in  developing  indicators  had  been 
confined  to  the  more  easily  measurable  efficiency  and  workload  targets. 
The  report  concluded  that  the  development  of  performance  measures  to 
appropriately  measure  outcomes  would  require  prolonged  effort  by 
government  agencies,  and  that  developing  useful  performance  indicators 
would  be  a  challenging  task  (31:29). 

In  the  case  of  the  British  Government,  a  1986  Treasury  Report 
defined  budgeting  as: 

...  a  means  for  delivering  value  for  money  against  a  background  of 
aims,  objectives  and  targets  ...  Budgeting  will  only  fully  realize 
its  full  potential  if  there  are  strong  connections  between  budgets 
outputs  and  results.  (18:8) 

Based  on  case  studies  issued  by  the  British  Treasury,  indications 
were  that  behavior  had  fallen  short  of  this  ideal,  and,  in  many  areas, 
objectives  had  not  been  expressed  with  sufficient  precision  to  allow 
assessment  of  whether  the  desired  objectives  have  been  achieved.  These 
performance  measurement  problems  had  been  noted  by  the  British 
Government  in  its  response  to  a  critique  of  the  British  Financial 
Management  Initiatives  by  the  Parliament's  Public  Accounts  Committee 
(31:27;  15). 

PMB,  TOC  and  the  RAAF  Performance  Msasurmaent  Environment 

The  literature  review  also  covered  references  pertaining  to 
Program  Management  and  Budgeting,  the  Theory  of  Constraints,  and  the 
RAAF  performance  measurement  environment.  However,  as  the  researcher 
believed  that  the  review  of  these  topics  also  involved  part  of  the 
research  analysis,  this  literature  review  information  was  included  in 
Chapter  IV. 

SuBsaary 

Chapter  II  exaunined  traditional  management  accounting  approaches. 
Activity  Based  Costing,  the  trend  towards  non-f inancial  performance 
measures,  and  the  unique  problems  associated  with  performance 
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measurement  in  not-for-profit  and  government  organizations.  As  a  result 
of  the  examinations  the  following  observations  were  provided  from  the 
literature: 

a.  The  traditional  accounting  model  was  not  always  suitable  in 
today's  operating  environment. 

b.  Activity  Based  Costing  provided  a  better  rearrangement  of 
financial  accounting  data,  but  did  not  have  the  capacity  to 
provide  timely  performance  measures  for  process  control. 

c.  There  was  an  increasing  trend  towards  the  use  of  non-f inancial 
performance  measures  for  organizations  pursuing  a  continual 
process  improvement  objective.  However,  there  were  no  definitive 
answers,  and  a  challenge  existed  to  develop  measures  that  suit 
each  organization. 

d.  The  unique  challenges  facing  government  organizations  in  the 
design  of  their  performance  measures  showed  that  performance 
measurement  for  these  organizations  was  not  a  trivial  problem. 

The  literature  indicated  that  there  is  a  pressing  need  for 
research  and  experimentation  in  the  development  of  performance 
measures  for  non-for-profit  and  government  organizations. 

Based  on  the  review  of  this  literature,  there  was  no  currently 

obvious  foundation  on  which  to  base  a  set  of  performance  measures  for 
the  RAAF.  Therefore,  pursuing  this  research  was  a  worthwhile  objective. 
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ZZZ.  ttethodology 


Owrvlaw 

To  propose  a  suitable  performance  measurement  concept  for  the 
Royal  Australian  Air  Force  (RAAF),  the  "critical  theory”  and  "action 
research”  methodologies  to  research  were  employed.  In  addition,  a  case 
study  was  employed  to  develop  a  set  of  measures  to  be  used  as  the  basis 
for  the  discussion  of  the  concept.  The  methodology  resulted  in  four 
phases  of  research  being  undertaken. 

Phase  1  determined  the  requirements  that  the  RAAF  has  for  its 
future  performance  measurement,  based  on  the  requirements  of  the  PMB 
system  adopted  by  the  Australian  000. 

Phase  2  involved  a  review  of  the  performance  measurement 
techniques  used  in  the  Theory  of  Constraints  (TOC).  The  aim  of  the 
review  was  to  provide  a  framework  for  the  development  of  a  set  of 
performance  measures  for  the  RAAF  Strike/Reconnaissance  Force  Element 
Group  (FEG).  These  measures  provided  the  foundation  for  the  case  study. 

Phase  3  involved  the  development  of  the  performance  measures  for 
the  Strike/Reconnaissance  FEG.  The  development  provided  a  case  study 
demonstrating  the  manner  in  which  the  TOC  could  be  adapted  to  the  RAAF. 

In  Phase  4,  a  comparison  of  the  proposed  performance  measurements 
against  the  criteria  determined  in  Phase  1  was  made.  The  aim  of  the 
comparison  was  to  determine  whether  the  proposed  methodologies  satisfied 
the  guidelines  set  out  by  PMB. 

The  Critical  Theory  and  Action  Research  J^proaches  to  Research 

The  critical  theory  approach  to  research  is  based  on  an  attempt  to 
facilitate  change,  rather  than  being  based  on  the  derivation  of  "natural 
laws"  as  found  in  the  empirical-analytical  science  approach  to  research. 
In  pursuing  a  change  in  an  organizational  process,  the  critical  theory 
explicitly  recognizes  that  the  researcher  has  a  motivating  interest  in 
conducting  the  research.  Thus,  the  researcher  must  identify  the  fact 
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that  hie  observations  were  not  value-free,  or  totally  objective  in  the 
empiric-analytic  sense  of  objectivity  (34:336). 

As  Steffy  and  Grimes  stated: 

Critical  theory  assumes  that  the  "worldview”  of  the  researcher 
constitutes  a  third  level  construct.  Theorists  and  researchers 
are  not  assumed  to  be  objective  and  value  free,  but  are  seen  as 
laden  with  beliefs  and  values  obtained  through  training,  peer 
group  influences,  and  the  goals  and  structures  of  the  research 
enterprise  itself.  (34:328) 

Critical  theory  requires  a  commitment  to  change,  and  it  is  this 
commitment  to  change  which  most  clearly  distinguishes  critical  theory 
from  other  approaches.  The  researcher  is  not  a  passive  observer  or  even 
a  participant.  Instead,  the  researcher  is  viewed  as  a 
"reconstructionist",  or  even  an  "activist”.  The  job  of  the  researcher 
is  to  increase,  for  the  members  of  the  organization  being  analyzed,  the 
understanding  of  their  current  condition,  and  subsequently,  establish 
conditions  for  organizational  change.  The  researcher  is  defined  as  a 
catalyst  whose  responsibility  is  to  ensure  that  the  conditions  were 
present  for  organizational  change.  The  conclusions  reached  in  such 
research  should  result  in  the  recommendation  of  a  pragmatic  approach  to 
changing  the  organization's  behavior  (34:333). 

Similarly,  the  action  research  approach  is  problem  oriented, 
having  the  objective  of  providing  a  result  which  can  be  directly  used  by 
the  organization  being  researched.  According  to  Sommer,  tne 
relationship  between  the  researcher  and  the  client  shifts  from  that  of 
the  academic  model  of  separation  of  the  client  organization  from  the 
results,  and  eventual  "trickle  down”  of  results  through  professional 
journals,  to  that  of  the  direct  collaboration  of  the  researcher  with  the 
client  organization.  Action  research  is  inherently  client  directed  as 
the  problem  formulation  and  subsequent  research  steps  are  taken  using 
the  client,  who  is  the  potential  user  of  the  research  results,  as  the 
basis  of  the  research  (33:197). 

As  the  researcher  in  this  research  paper  was  to  be  actively 
involved  in  developing  future  performance  measures  for  the  RAAF,  the  use 
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of  the  critical  theory  and  action  research  approaches  was  considered  to 
be  the  most  appropriate  framework  for  the  research.  The  researcher 
acknowledged  that  he  was  attempting  to  become  a  catalyst  for  change,  and 
that  the  research  was  centered  on  providing  a  solution  to  a  problem  that 
faced  the  organization  in  which  the  researcher  was  working. 

The  Use  of  the  Case  Study 

As  described  by  Yin,  the  case  study  is  a  legitimate  form  of 
research,  and  is  but  one  of  several  ways  in  which  research  can  be 
conducted.  In  particular,  case  studies  can  be  the  preferred  strategy 
when  "how"  or  "why"  questions  are  being  posed,  when  the  investigator  has 
little  control  over  events,  and  when  the  focus  of  the  research  is  on  a 
contemporary  phenomenon  within  a  real-life  context  (36:13,18-20). 

Case  studies,  as  a  form  of  research  contribute  uniquely  to  our 
knowledge  of  individual,  organizational,  social,  and  political 
phenomena.  The  distinctive  need  for  case  studies  has  its  origins  in  the 
desire  to  understand  complex  social  phenomena,  with  the  case  study 
allowing  an  investigation  to  retain  the  holistic  and  meaningful 
characteristics  of  real-life  events  —  such  as  found  in  organizational 
and  managerial  processes  (36:14). 

According  to  Yin,  the  case  study  strategy  can  be  used  for  three 
purposes  —  exploratory,  descriptive  or  explanatory  research.  In 
particular,  the  strategy  may  be  used  to  explore  those  situations  in 
which  the  subject  of  the  research  being  evaluated  has  no  clear,  single 
set  of  outcomes.  This  is  especially  so  when  the  case  study  is  examining 
contemporary  events  where  the  relevant  behaviors  cannot  be  manipulated. 
In  these  cases,  Yin  stated  that  the  case  study  was  the  preferred  method 
for  conducting  the  research  (36:15-16,19,25). 

A  major  rationale  for  using  a  single  case  study  is  when  the  study 
represents  a  revelatory  case.  This  situation  arises  when  the 
investigator  identifies  an  opportunity  to  analyze  a  phenomenon 
previously  inaccessible  to  investigation.  Further  insights  into  the 


single  case  can  be  obtained  when  attention  is  being  also  given  to  a 
subunit  or  subunits  within  the  system  being  investigated.  Finally,  Yin 
states  that  case  studies  do  not  always  need  to  include  direct,  detailed 
observations  as  a  source  of  evidence  (3:25,43,47). 

The  case  study  approach  was  considered  to  be  applicable  to  the 
research  being  conducted  on  performance  measures  in  the  RAAF.  The 
research  was  exploratory  in  nature,  and  addressed  the  ideas  expressed  in 
a  contemporary  set  of  management  concepts.  Having  only  been  developed 
during  the  late  1980s,  the  concepts  of  the  Theory  of  Constraints  have 
not  been  widely  examined,  especially  in  the  government  environment.  A 
search  of  the  literature  revealed  no  mention  of  the  detailed  use  of 
these  concepts  in  the  government  environment. 

In  addition,  with  the  RAAF  only  beginning  to  examine  performance 
measures  in  the  light  of  the  PMB  initiative,  there  was  no  data  to  draw 
"from  the  field".  The  concepts  involved  in  the  research  were  in  fact 
postulating  a  new  set  of  measures  for  use  in  the  future.  In  this  vein, 
the  research  was  revelatory,  as  it  showed  the  possibility  of  using  a  new 
approach  for  developing  performance  measures  in  the  RAAF.  The  new 
environment  created  by  PMB,  and  the  use  of  new  concepts  also  indicated 
that  there  would  be  no  clear  cut  single  set  of  outcomes  relating  to  the 
choice  of  performance  measures  to  be  used  in  the  RAAF.  Thus,  the 
combination  of  factors  surrounding  the  research  made  the  case  study  the 
ideal  vehicle  for  the  conduct  of  the  research,  and  complemented  the 
critical  theory  and  action  research  approaches  employed  in  the  research. 

Specific  Research  Methodology 

To  accomplish  the  research  objective,  four  investigative  questions 
were  formulated.  From  these  questions,  specific  methodologies  were 
developed  to  provide  the  data  to  answer  the  four  investigative 
questions. 

The  four  investigative  questions,  and  their  associated 
methodologies,  were: 
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Investigative  Question  1 .  What  types  of  performance  measurement 
indicators  required  in  the  PMB  system  adopted  by  the  Australian  DOO,  and 
what  attributes  do  these  measures  require? 

Methodology  la.  Examine  the  performance  measurement  rec[uirements 
of  the  Department  of  Defence  Progreun  Management  Budgeting  System. 

Investigative  Question  2.  How  can  the  performance  measurement 
concepts  of  the  TOC  be  adapted  to  a  government  organization? 

Methodology  2a.  Conduct  a  review  of  the  TOC  performance 
measurement  concepts. 

Methodology  2b.  Using  the  TOC  concepts  based  on  the  for-profit 
organization,  apply  the  concepts  to  a  government  organization. 

Investigative  Question  3.  What  is  an  example  of  the  TOC 
methodology  as  applied  to  a  RAAF  PEG? 

Methodology  3a.  Identify  the  operating  environment  for  one  PEG. 

Methodology  3b.  Using  the  concepts  obtained  from  answering 
Question  3,  formulate  a  set  of  global  performance  measures  which  can  be 
used  by  that  PEG  Commander  to  monitor  the  PEG'S  performance. 

Investigative  Question  4.  Does  the  PEG  example  satisfy  the 
performance  measurement  requirements  of  the  PMB  process? 

Methodology  4a.  Compare  the  PEG  performance  measures  developed 
in  the  answer  to  Question  3,  to  the  guidelines  formulated  in  answer  to 
Question  1. 

Research  Phase  1 

Research  Phase  1  encompassed  Investigative  Question  1,  and 
Research  Methodology  1.  The  aim  of  this  phase  was  to  obtain  the  basic 
requirements  for  the  performance  measurement  system  sought  by  the  RAAP, 
thereby  providing  the  criteria  against  which  a  suggested  methodology 
could  be  compared. 

Research  Phase  2 

Research  Phase  2  involved  Investigative  Questions  2  and  Research 
Methodologies  2a  and  2b.  Phase  2  reviewed  the  performance  measurement 
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aspects  of  the  TOC,  explaining  the  rationale  behind  the  performance 
measures.  As  the  TOC  concepts  were  developed  in  a  profit  making 
environment,  the  concepts  needed  to  be  slightly  modified  to  adapt  them 
to  a  government  organization.  The  TOC  was  chosen  as  the  basis  for  the 
performance  measures  because  the  concepts  in  the  TOC  were  only  derived 
in  the  mid  1980' s,  and  had  not  been  evaluated  in  the  Australian 
Government  environment.  The  research  goal  was  see  whether  the  TOC 
concepts  were  suitable  for  use  in  the  RAAF. 

Research  Phase  3 

In  Research  Phase  3,  Investigative  Question  3,  using  Research 
Methodologies  3a  and  3b,  was  addressed.  This  phase  required  the 
selection  of  one  PEG  and  the  identification  of  the  structure  of  a  set  of 
performance  measures  for  that  PEG,  the  objectives  of  the  PEG,  and  where 
these  objectives  fit  into  the  overall  goals  for  the  RAAF.  Based  on  this 
operating  environment,  a  set  of  performance  measures  were  developed 
using  the  concepts  of  the  TOC.  As  the  Strike/Reconnaissance  PEG  was  the 
first  RAAF  PEG  to  incorporate  its  own  Logistics  Support  Management 
Squadron,  and  as  the  other  FEGs  were  to  follow  this  trend,  the 
Strike/Reconnaissance  FEG  was  chosen  to  be  the  basis  of  the  case  study. 

Research  Phase  4 

Research  Phase  4  dealt  with  Investigative  Question  4,  and  Research 
Methodology  4a.  This  was  the  analysis  phase  of  the  research  where  the 
performance  measurements  constructed  in  Phase  3  were  compared  to  the 
guidelines  developed  in  Research  Phase  1.  The  comparison  detailed  those 
areas  where  the  methodology  met  the  guidelines. 

Methodology  Limitations 

The  problem  focus  for  the  research  was  exploratory  in  nature.  The 
objective  was  to  examine  whether  the  performance  measurement  concepts  of 
the  TOC  could  be  applied  to  a  RAAF  FEG.  Unfortunately,  time  and 
resource  constraints  did  not  permit  the  research  to  be  carried  "in  the 
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Data  collection  to  answer  the  investigative  questions  was  provided 
through  literature  research.  Thus,  the  research  was  descriptive  in 
format,  with  no  formal  cause  and  effect  relationships  being  examined. 

With  potential  RAAF  FEG  Commanders  having  little  experience  with 
cost  management  and  performance  measurement,  the  research  was  unable  to 
elicit  the  management  requirements  of  these  Commanders.  Therefore,  the 
research  relied  on  DOD  policy  makers'  guidance  on  the  requirements  for 
performance  measurement  as  detailed  in  the  PMB  Reference  Manual  (5). 

The  performance  measures  developed  in  the  research  were  only  provided  as 
a  basis  for  discussion,  and  modifications  should  be  made  to  the  measures 
as  FEG  Commanders  become  more  familiar  with  their  cost  management  role 
and  develop  an  understanding  of  the  performance  measurement  requirements 
for  their  areas  of  responsibility.  The  amendments  to  the  system  should 
essentially  be  to  the  manner  in  which  the  performance  variables  are 
operationally  defined,  rather  than  amendments  to  the  underlying 
performance  measurement  methodology. 

Sunanary 

Chapter  III  explains  the  approach  taken  in  conducting  the  research 
and  details  the  methodologies  required  to  provide  the  answers  to  the 
investigative  questions  posed  in  Chapter  I.  The  research  used  the 
critical  theory  and  action  research  approaches  to  conduct  the  research, 
utilizing  a  case  study  as  the  vehicle  to  present  the  concepts  examined. 
The  methodologies  used  were:  the  examination  of  the  PMB  guidelines  for 
determining  the  format  of  a  performance  measure;  a  review  of  the 
performance  measurement  concepts  of  the  Theory  of  Constraints,  and  their 
adaption  to  a  government  organization;  the  development  of  a  set  of 
performance  measures  for  the  RAAF  Strike/Reconnaissance  FEG;  and  a 
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comparison  of  those  performance  measures  against  the  guidelines  for 
performance  measures  provided  in  the  PMB  Reference  Manual.  The 
limitations  of  the  research  methodology  were  also  discussed. 
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IV.  Findings  and  Analysis 


Ovarviaw 

This  chapter  details  the  results  of  the  examination  of  each  of  the 
investigative  questions.  The  chapter  provides  the  Program  Management 
and  Budgeting  (PMB)  guidelines  by  which  any  performance  measure  adopted 
by  the  RAAF  should  be  assessed,  discusses  how  the  performance 
measurement  concepts  of  the  Theory  of  Constraints  can  be  adapted  to  the 
non-profit  environment,  discusses  and  develops  a  set  of  performance 
measures  for  the  RAAF  Strike/Reconnaissance  FEG,  and  compares  the 
performance  measures  developed  to  the  PMB  guidelines. 

Program  Management  and  Budgeting 

The  examination  of  the  requirements  of  PMB  was  undertaken  to 
provide  an  answer  to  the  first  investigative  question. 

Investigative  Question  1.  What  type  of  performance  measurement 
indicators  are  required  in  the  PMB  system  adopted  by  the  RAAF? 

As  the  PMB  system  adopted  by  the  Australian  Department  of  Defence 
(DOD)  must  form  the  cornerstone  of  any  performance  measurement  system 
used  within  the  RAAF,  an  overview  of  the  PMB  environment  was  obtained. 

PMB  had  been  developed  as  a  comprehensive  management  philosophy  to 
focus  attention  on  the  assessment  of  the  effectiveness  of  programs 
against  objectives,  and  on  the  efficiency  of  resource  use.  A  prime 
tenet  of  PMB  was  that  managers'  knowledge  of  their  effectiveness  and 
efficiency  was  not  sufficient  —  they  must  be  able  to  demonstrate  their 
efficiency  and  effectiveness  to  others.  Further,  PMB  was  much  more  than 
just  an  accounting  process,  and  was  developed  to  allow  performance  and 
effectiveness  to  be  expressed  in  both  financial  and  non-f inancial  terms 
(5:1-1, 1-9). 

The  stated  thrust  of  PMB  was  to  concentrate  on  a  more  outcome 
oriented  and  strategic  approach  to  decision  making.  The  PMB  process 
sought  to  improve  management  efficiency  and  effectiveness  through: 


a.  the  setting  of  objectives; 

b.  the  development  of  hierarchical  prograun  structures  to 
achieve  those  objectives, 

c.  the  allocation  of  resources  to  those  programs,  and 

d.  the  evaluation  of  program  performance.  (5:1-3) 

With  these  goals  in  mind,  the  following  design  principles  were 
included  in  the  DOD  PMB  system: 

The  (PMB)  structure  should  support  assessment  of  performance  in 
terms  of  outcomes.  Since,  in  Defence,  the  major  outcomes  are 
combat  forces  at  appropriate  degrees  of  readiness,  the  emphasis  is 
on  reporting  in  terms  of  FEGs. 

The  emphasis  on  visibility  of  outcomes  means  that  reporting  and 
information  systems  need  to  be  developed.  These  will  need  to 
support  programs  and  FEGs. 

FEGS  performance  and  resource  consumption  will  need  to  be  captured 
for  internal  analysis,  and  also  to  develop  performance  indicators 
for  public  presentation.  (5:4-2) 

Thus,  a  major  requirement  of  the  DOD  implementation  of  PMB  was  the 
need  for  visibility  of  outcomes,  both  for  internal  and  external 
purposes,  with  a  treatment  of  performance  in  terms  of  FEGs  (4:6). 

The  DOD  recognized  that  it  would  be  inappropriate  to  require  all 
Program  Managers  to  adhere  to  a  standard  level  of  reporting,  since 
complexities  of  intra-program  arrangements,  performance  indicators  and 
resource  implications  differed  substantially  between  programs.  Thus, 
the  independent  development  of  performance  measures  for  each  FEG  was 
acceptable  (5:5-4). 

Under  the  DOD  PMB  system,  the  management  of  the  DOD  was  broJcen  up 
into  eight  programs.  One  of  these  programs  is  the  RAAF,  with  the  Chief 
of  the  Air  Staff  (CAS)  acting  as  the  program  manager.  The  program 
manager's  responsibilities,  among  others,  were  to: 

a.  develop  strategies,  implement  plans,  activity  targets,  activity 
indicators  and  other  measures  of  assessment  of  performance  against 
endorsed  guidance; 

b.  monitor  and  review  program  performance  in  terms  of  outcomes 
against  objectives  and  the  efficient  use  of  resources; 
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c.  provide  and  maintain  information  appropriate  to  the  effective  and 
efficient  management  of  sub-programs  and  subordinate  elements;  and 

d.  develop  and  refine  resource  attributions,  objectives,  performance 
and  outcome  information  for  lower  level  program  elements, 
particularly  for  evaluation  and  program  development  purposes 
(5:2-7). 

These  defined  responsibilities  reinforced  the  PMB  focus  on  the 
effective  achievement  of  results  against  objectives  and,  on  the 
efficient  use  of  resources,  rather  than  focusing  solely  on  the 
achievement  of  expenditure. 

When  examining  the  proposals  for  Defence's  sub-program  structure, 
the  Defence  Force  Development  Committee  agreed  that  the  sub-program 
structures  in  each  program  should  support  assessments  of  performance  in 
terms  of  outcomes.  For  the  three  armed  Services,  the  major  outcomes  of 
their  individual  programs  were  combat-ready  forces.  Consequently,  the 
emphasis  was  to  be  placed  on  PMB  planning,  management  and  performance 
reporting  in  terms  of  FEGs  as  identified  in  the  Chief  of  the  Defence 
Force's  (CDF's)  Operational  Readiness  Directive  (CORD)  (5:4-6). 

FEGs  were  seen  as  the  key  factor  in  relating  the  chain  of 
command/line  management  oriented  program  structure  to  results  and 
outcomes.  Consequently,  it  was  envisaged  that  FEGs  would: 

a.  provide  the  basis  for  public  presentation  of  performance 
information  on  outcomes  of  the  Defence  Program,  and 

b.  become  the  basis  for  corporate  planning  and  review  of 
ongoing  activities  and  expenditure.  (5:4-6) 

Finally,  the  DOD  PMB  process  would  be  based  on  guidance  from  the 

Government  and  on  guidance  from  the  CDF.  An  example  of  the  planning 

guidance  to  be  used  was  the  classified  CORD.  Such  guidance  would 

provide  the  planning  objectives  against  which  program  managers  could 

test  the  outcomes  of  their  processes,  their  achievements,  the 

appropriateness  of  exercises  and  training  programs,  the  level  of 

stoclcholdings,  personnel  availability  etc  (4:9). 


FMB  Guidsllncs  for  a  Parfonnanc*  Maasuramant  System 

Based  on  the  PMB  Reference  manual  and  the  foregoing  discussion,  12 
guidelines  were  extracted  for  developing  performance  measures  for  the 
RAAF.  These  guidelines  were: 

a.  The  performance  measures  must  assess  the  effectiveness  of  programs 
against  objectives. 

b.  The  performance  measures  must  assess  the  efficiency  of  resource 
use. 

c.  Performance  and  effectiveness  could  be  expressed  in  both  financial 
and  non-f inancial  terms 

d.  The  performance  measures  should  concentrate  on  an  outcome  oriented 
and  strategic  approach  to  decision  making. 

e.  Performance  reports  were  to  be  in  terms  of  FBGs. 

f.  Performance  indicators  may  differ  between  FEGs. 

g.  Different  methods  of  reporting  can  be  undertaken  at  different 
management  levels. 

h.  Performance  measurement  was  to  be  achieved  using  cost  effective 
means . 

i.  Attribution  of  resources  to  FEGs  was  to  assist  analysis,  program 
evaluation  and  presentation  rather  than  provide  accounting 
accuracy. 

j.  There  was  to  be  an  emphasis  on  the  effective  achievement  of 
outcomes  against  objectives,  and  on  the  efficient  use  of 
resources,  rather  than  solely  emphasizing  expenditure  achievement. 

k.  PMB  requirements  should  not  impose  a  burden  on  operational 
effectiveness  or  impose  a  high  resource  overhead  on  combat  units. 

l.  FEGs  needed  to  participate  in  the  self-evaluation  of  their 
individual  performance. 

In  designing  a  set  of  performance  measures,  some  consideration 
also  needed  to  be  given  to  the  method  to  develop  these  performance 
measures.  For  this  research,  guidelines  were  adopted  from  the  AFSC 
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handbook,  entitled  The  Metrics  Handbook  (1).  These  guidelines  were 
summarized  in  Appendix  B. 

A  Review  of  TOC  -  The  Profit  World 

The  research  concentrated  on  developing  performance  measurements 
based  on  the  measurement  concepts  suggested  by  E.  Goldratt  in  his  Theory 
of  Constraints  (TOC).  One  of  Goldratt 's  main  approaches  to  performance 
measurement  was  the  use  of  measures  based  on  his  definitions  of 
throughput  (T),  inventory  (I),  and  operating  expense  (OE). 

Before  performance  measurements  based  on  T,  I,  and  OE  could  be 
developed,  these  elements  needed  to  be  defined,  and  the  derived  metrics 
needed  to  be  described.  The  initial  review  of  TOC  examined  some 
critical  concepts  in  the  production/  capacity  component  of  TOC,  with  the 
examination  being  conducted  on  the  basis  of  a  profit  seeking 
organization.  Having  described  TOC  measures  in  commercial  terms  for  the 
sake  of  exposition,  a  transfer  of  the  concepts  to  the  not-for-profit 
world  was  undertaken.  The  transfer  involved  the  substitution  of  the 
goals  of  a  non-profit  government  organization  in  the  place  of  the  profit 
goal  of  the  commercial  organization. 

The  result  of  this  review  provided  the  answer  to  the  second 
investigative  question: 

Investigative  Question  2.  How  can  the  performance  measurement 
concepts  of  the  TOC  be  adapted  to  a  government  organization? 

In  the  production/capacity  concept  of  TOC,  every  aspect  of  the 
organization  was  designed  to  focus  on  the  goal  of  the  organization.  In 
the  profit  world,  Goldratt  defined  the  goal  of  the  firm  to  be  to  make 
money,  both  now  and  in  the  future.  To  this  end,  performance  measures 
must  reflect  the  organization's  progress  to  achieving  this  goal,  and  all 
measures  should  influence  employee  behavior  towards  achieving  that  goal 
(16:19-26) . 

In  TOC,  T,  I  and  OE  had  different  definitions  than  those  used  in 
traditional  cost  performance  measures.  These  new  definitions  were: 
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a.  Throughput  —  the  rate  at  which  the  system  generated  money 
through  sales,  e.g.  dollars  generated  per  day,  dollars  generated 
per  hour,  etc. 

b.  Inventory  —  All  the  money  the  system  had  invested  in  purchasing 
things  the  system  intends  to  sell,  e.g.  raw  materials,  productive 
assets  (productive  assets  can  be  sold  at  a  later  date  by  the 
company  to  produce  money). 

c.  Operating  Expense  —  All  the  money  the  system  had  spent  in 
turning  inventory  into  throughput,  e.g.  direct  labor,  indirect 
labor,  depreciation  of  assets,  and  other  overheads  (16:19-29). 

These  three  measures  provide  answers  to  the  questions:  at  what  rate  is 
the  company  generating  money;  how  much  money  is  captured  within  the 
company;  and  how  much  has  to  be  spent  to  operate  the  company  (16:16-18)? 

There  were  four  important  interpretations  attached  to  these 
definitions.  First,  throughput  was  not  achieved  until  a  product  of  the 
company  had  permanently  left  the  company.  Thus,  in  the  commercial 
sector,  throughput  is  not  achieved  until  a  product  produced  by  the 
company  had  been  sold,  such  that  the  products  are  not  subject  to  return 
to  the  company.  For  example,  finished  goods  inventories,  consignment 
stocks,  serviceable  spares,  etc,  could  not  be  considered  to  be 
throughput  because  they  had  not  been  converted  to  money  through  a  cash 
transfer  from  the  customer  to  the  manufacturer.  Care  must  be  taken  in 
determining  when  the  throughput  was  actually  achieved,  and  a  sale  should 
be  recorded  only  when  an  irrevocable  transaction  has  occurred  with  the 
final  consumer  (16:21-22). 

Second,  throughput  was  calculated  as  the  selling  price  minus: 

a.  amounts  paid  to  vendors  for  items  that  went  into  the  product  sold, 
no  matter  when  these  items  vrere  actually  bought;  and 

b.  the  costs  of  services  provided  by  organizations  outside  the 
organization's  system  —  commissions  to  external  salesmen, 
subcontracting  expenses,  customs  duties,  transport  charges  for 
non-organization  owned  transport,  etc. 
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Thus,  throughput  was  the  actual  outcome  generated  by  the  company, 
which  in  this  case  was  the  generation  of  additional  cash  —  the 
difference  between  revenue  and  the  direct  inventory  costs  for  the 
product  (not  including  operating  expenses). 

Third,  inventory  encompassed  all  raw  materials  and  other  materials 
input  into  the  system's  process.  The  "other  materials"  included  items 
that  have  been  traditionally  referred  to  as  assets  such  as  buildings  and 
machinery.  These  items  were  recorded  at  their  original  cost  plus  any 
other  inventory  investment  attached  to  them.  For  example,  a  finished 
good  in  a  warehouse  had  the  following  cost:  the  actual  dollar  amounts 
(paid  to  vendors  for  raw  materials  and  other  externally  supplied  goods 
and  services)  required  to  produce  the  finished  good.  This  cost  did  not 
include  costs  related  to  the  "added  value"  provided  by  the  system. 

Thus,  direct  labor  and  depreciation  were  not  attributed  to  the  product, 
but  were  treated  as  a  period  operating  expense.  The  new  definition 
contrasts  with  the  traditional  costing  methods  which  value  finished 
goods  using  such  techniques  as  FIFO  inventories  and  absorption  costing 
(16:23) . 

Finally,  Operating  Expense  included  all  money  spent  to  convert  the 
inventory  into  throughput.  This  included  direct  labor,  indirect  labor, 
overheads,  and  depreciation  of  assets. 

The  importance  of  these  new  definitions  was  that,  by  adopting 
definitions  for  throughput,  inventory,  and  operating  expenses  based  on 
the  principles  in  the  Theory  of  Constraints,  there  was  no  need  to 
underta)ce  the  time  consuming  task  of  preparing  costs  based  on  absorption 
costing  or  other  overhead  cost  allocation  techniques. 

Further,  as  all  these  elements  were  denominated  in  dollars,  the 
combination  of  these  three  measures  provided  the  following  performance 
measurements : 

a.  Net  Profit:  NP  *  T  -  OE; 

b.  Return  on  Investment:  ROI  «  NP/I; 

c.  Inventory  turnover  «  T/I;  and 
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d.  Productivity  »  T/OE  (16:32). 

Using  these  measures,  management  could  see  that  the  way  to  improve 
performance  was  to  increase  throughput,  decrease  inventory,  or  decrease 
operating  expense.  Of  course,  in  many  circumstances,  increasing 
throughput  could  require  an  increase  in  operating  expense.  However,  the 
manager  could  immediately  see  that  if  the  increase  in  throughput  is 
greater  than  the  increase  in  operating  expense  then  the  decision  to 
increase  throughput  would  be  a  correct  decision. 

TOC  in  the  Government  Environswnt 

In  the  government  environment,  the  goal  was  not  necessarily  to 
make  money.  Therefore,  money  gained  from  sales  was  not  a  critical 
activity,  although  the  generation  of  money  may  have  been  a  necessary 
condition  for  continuing  operations.  In  the  government  organization  the 
goal  was  something  other  than  earning  a  profit  for  its  owners,  and 
usually,  that  organization's  goal  was  to  provide  services.  Many  of 
these  organizations  are  designed  to  always  breakeven,  e.g.  the 
Department  of  Defence  (2:31). 

Thus,  the  government  organization  could  not  necessarily  define 
throughput  in  terms  of  money  generated  by  the  organization,  and 
alternative  definitions  for  T,  I  and  OE  were  required.  For  the 
government  organization,  these  terms  could  be  defined  as  follows: 

a.  Throughput  —  the  rate  at  which  the  system  achieved  its  goal(s). 

b.  Inventory  —  everything  the  system  had  invested  to  produce 
throughput . 

c.  Operating  Expense  —  All  the  money  the  system  had  spent  in 
turning  inventory  into  throughput  (35). 

The  concept  of  throughput  also  needed  to  be  modified  from  the 
concept  found  in  the  commercial  version  of  the  TOC.  In  the  comnercial 
world,  the  rate  of  throughput  refers  to  the  generation  of  money  in  a 
given  period  of  time,  for  example,  the  money  generated  in  one  year  of 
production.  An  increase  in  throughput  would  be  the  result  of  the 
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generating  more  money  in  a  one  year  time  span,  with  the  potential  for 
further  increases  in  throughput  being  theoretically  unlimited. 

However,  in  many  government  organizations,  ceilings  are  placed  on 
the  organization's  budget,  or  on  the  amount  of  throughput  that  can  be 
achieved,  e.g.  the  number  of  flying  hours  which  can  be  flown  by  the  RAAF 
has  a  ceiling  fixed  by  higher  Defence  management.  Therefore,  the 
emphasis  in  the  government  environment  would  be  to  achieve  the  required, 
fixed  throughput  with  the  highest  degree  of  effectiveness  and 
efficiency.  Throughput  then  becomes  the  rate  that  effective  output  is 
achieved  per  unit  of  output  produced  by  the  organization.  The  desired 
level  of  throughput  becomes  the  achievement  of  100%  of  the  output 
defined  in  terms  of  the  organization's  objectives. 

Where  profit  provided  both  a  measure  of  effectiveness  and  a 
measure  of  efficiency  for  the  commercial  organization,  such  a  single 
overall  measure  of  performance  did  not  exist  for  the  government 
organization.  However,  in  the  government  organization  it  may  be 
feasible,  and  useful,  to  develop  and  classify  performance  measures 
relating  either  to  effectiveness,  or  to  efficiency.  A  combination  of 
effectiveness  and  efficiency  measures  could  then  be  used  to  assess  the 
organization's  performance.  These  measures  would  not  be  based  on  money 
as  the  only  unit  of  measurement,  as  the  organization’s  output  could  not 
be  measured  in  monetary  terms  (2:6). 

Since  TOC  was  originally  based  on  the  commercial  world,  this 
research  had  to  establish  new  criteria  to  monitor  performance.  The 
following  performance  measures  were  therefore  developed: 

a.  Effectiveness  —  The  rate  defined  within  the  definition  of 
throughput.  Performance  could  be  monitored  by  establishing  a 
Statistical  Process  Control  (SPC)  control  chart  which  would 
indicate  changes  in  performance,  and  could  be  used  to  set 
quantifiable  goals  for  the  organization. 

b.  Efficiency  —  there  are  two  efficiency  measures  which  can  be  used: 
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(1)  throughput  divided  by  inventory  to  provide  throughput  per 
dollar  invested  in  inventory;  and 

(2)  throughput  divided  by  operating  expense  to  provide  a  measure 
of  productivity. 

These  efficiency  measures  could  be  monitored  using  appropriate  run 
charts. 

The  performance  measures  represented  "global  measures"  for  the 
organization.  However,  their  construction  allowed  for  the  breakdown  of 
the  measures  for  use  by  sub-units  of  the  organization.  This  breakdown 
was  examined  in  greater  detail  during  the  development  of  the  performance 
measures  for  the  RAAF  Strike/Reconnaissance  FEC. 

The  RAAF  FEG  Performance  Measurement  Environment 

The  exeunination  of  the  RAAF  FEG  cost  management  environment,  and 
the  development  of  performance  measures  for  the  Strike/Reconnaissance 
FEG  provided  the  answer  to  the  third  investigative  question: 

Investigative  Question  3.  What  is  an  exaunple  of  the  TOC  concepts 
as  applied  to  a  RAAF  FEG? 

To  begin  to  use  the  concepts  of  TOC  performance  measurements  in 
the  RAAF  environment,  that  environment  was  examined.  First,  the  mission 
and  goals  of  the  RAAF  were  listed  at  Appendix  C.  Of  the  goals  listed, 
goals  la(iv),  lb,  2e,  and  3a  could  be  monitored  for  an  FEG  using  the 
measures  proposed  in  this  research. 

Discussion  of  RAAF  Goals.  Goal  la(iv)  sought  to  provide  quality 
logistics  support  for  all  RAAF  operations.  Such  a  goal  was  fairly  broad 
in  its  interpretation,  and  it  was  difficult  to  define  a  way  of  directly 
measuring  the  achievement  of  this  goal.  For  example,  how  is  quality 
logistics  support  to  be  operationally  defined,  and  then  subsequently 
measured?  In  this  situation,  the  goal  could  be  operationally  defined 
using  the  following  surrogate  measure. 

For  a  flying  squadron,  quality  logistics  support  could  be  defined 
as  that  level  of  logistics  support  which  allows  a  squadron  to  achieve 
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95%  (say)  success  in  launching  missions  at  the  scheduled  time,  with  the 
individual  aircraft  reliability  being  such  that  the  aircrew  was  able  to 
complete  its  mission  successfully.  A  surrogate  measurement  of  the 
outcomes  of  activities  directed  at  achieving  this  "operationalized"  goal 
could  be  to  measure  the  proportion  of  aircraft  in  the  squadron  which 
were  able  to  take-off  at  the  scheduled  time  and  for  which  the 
aircraft's  crew  were  able  to  successfully  undertake  their  scheduled 
mission. 

Goal  lb  sought  to  maintain  operational  readiness  by  achieving  the 
targets  specified  in  the  CORD.  In  the  absence  of  direct  access  to  the 
CORD,  owing  to  its  classified  nature,  the  objectives  of  the  CORD  could 
be  operationally  defined  in  a  similar  manner  to  the  operational 
definition  provided  for  goal  la(iv).  Thus,  the  outcomes  of  activities 
directed  at  achieving  both  goals  la(iv)  and  lb  could  be  measured  using 
the  same,  or  similar  surrogate  measures. 

Goal  2e  sought  to  ensure  timely  and  effective  maintenance  and 
logistic  support.  This  goal,  when  couched  in  such  broad  terms,  was  very 
similar  to  goal  la(iv).  While  a  direct  measure  of  the  required 
timeliness  and  effectiveness  of  maintenance  and  logistic  support  is  not 
readily  identifiable,  a  surrogate  measure  is.  If  maintenance  and 
logistics  support  was  to  be  timely  and  effective,  then  operational 
aircraft  should  be  able  to  take-off  at  their  scheduled  departure  time, 
and  the  aircraft  crew  should  be  able  to  successfully  undertake  their 
scheduled  missions.  Thus,  the  surrogate  measures  identified  for  goal 
la(iv)  was  also  a  suitable  surrogate  for  measuring  performance  against 
goal  2e. 

Finally,  goal  3a  sought  to  search  continuously  for  better  ways  to 
do  the  job.  Direct  measurement  of  the  attainment  of  this  goal  was 
difficult  to  achieve.  What  constituted  searching?  Are  the  number  of 
searches  to  be  counted  to  provide  a  measure  of  the  level  of  search?  In 
this  case,  a  surrogate  method  for  measuring  the  achievement  of  this  goal 
was  rec[uired.  One  possible  surrogate  was  to  monitor  the  trends 
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exhibited  by  the  performance  measures  used  to  measure  the  attainment  of 
the  other  RAAF  goals.  For  example,  the  surrogate  measure  defined  for 
goal  la(iv)  can  be  monitored  to  ascertain  whether  the  measurements 
observed  were  showing  a  trend  which  reflected  a  continuous  (an 
operational  definition  of  continuous  would  be  required)  improvement  in 
performance.  Regular  increases  in  effectiveness,  coupled  with  decreases 
in  operating  expenses  and  inventory,  could  be  taken  to  reflect  the 
achievement  of  the  goal  of  searching  continuously  for  better  ways  to  do 
the  job. 

From  the  foregoing  discussion,  the  need  to  be  able  to 
operationalize  the  RAAF  goals  became  clear.  Without  such  operational 
definitions,  and  their  associated  surrogate  measures,  measuring 
performance  became  next  to  impossible  in  the  RAAF  environment  where,  in 
peace  time,  the  end  product  of  all  RAAF  activity  was  an  intangible 
service  —  deterrence  against  aggression  by  foreign  forces,  and 
provision  of  an  insurance  policy  to  provide  protection  for  the  nation 
should  aggression  actually  be  applied  against  Australian  interests. 

The  next  major  factor  in  the  development  of  performance  measures 
for  the  RAAF  related  to  the  prospect  of  military  threat  against 
Australian  interests.  The  stated  Australian  military  environment  was 
that  there  were  no  discernable  major  military  threats  for  the 
foreseeable  future  (6:24-26).  The  future  management  emphasis  was 
therefore  to  be  based  on  peacetime  operations.  Accordingly,  the  outcome 
of  the  RAAF  program  became  the  RAAF  flying  hour  program,  which  takes 
into  account  the  CORO,  national  tasks,  and  approved  levels  of  training. 
This  flying  hour  progr2un  became  the  focus  for  managerial  attention  for 
monitoring  the  effectiveness  and  efficiency  of  RAAF  activities  (30:14). 

Thus,  the  goal  of  the  FEGs  could  be  considered  to  be  the 
achievement  of  their  flying  hour  program  in  the  most  effective  and 
efficient  manner  possible.  Therefore,  this  research  proposed  that  a 
definition  for  throughput  for  the  FEG  should  relate  to  these  flying 
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hours.  More  specifically,  a  first  operational  definition  for  this 
throughput  was; 

The  rate  at  which  the  FEG  achieves  "effective  hours”  per  scheduled 
flying  hour. 

The  definition  of  "effective  hours"  would  be: 

The  achievement  of  scheduled  flying  hours  for  a  mission,  such  that 
the  mission  took  place  on-time,  and  achieved  the  objectives  of 
the  mission. 

For  example,  assume  a  training  mission  was  planned  to  take  two 
hours  to  complete,  that  planned  take-off  was  at  lOOOhrs,  and  that  the 
mission  was  to  complete  a  photographic  run  over  an  army  exercise  area. 

If  the  mission  took  off  on  time,  and  completed  the  required  photography 
then  two  "effective  hours"  have  been  achieved.  However,  if  the  aircraft 
departed  late  (late  needs  to  be  operationally  defined  e.g.  at  least  10 
minutes  after  the  scheduled  take-off  time),  or  the  mission  could  not  be 
completed  due  to  equipment  failure  or  pilot  error,  then  two  scheduled 
effective  hours  have  not  been  achieved. 

The  on-time  requirement  acts  as  a  surrogate  for  assessing  the 
readiness  component  of  the  preparedness  goal,  and  the  achievement  of 
mission  objectives  indicates  the  capacity  to  complete  missions  the  FEG. 
The  combination  of  these  two  attributes  indicates  the  effectiveness  of 
the  FEG  to  complete  missions  in  the  timescales  for  which  the  missions 
were  planned.  The  penalty  inherent  in  deducting  the  total  scheduled 
flying  hours  from  the  performance  rating  reflects  a  weighting  for  loss 
of  effectiveness  on  longer  missions.  This  reflects  the  importance  of 
being  able  to  effectively  complete  the  longer  missions  required  to  be 
undertaken  by  a  strike/reconnaissance  force. 

To  provide  the  required  performance  measures,  operational 
definitions  of  inventory  and  operating  expense  were  also  required. 
Inventory  was  considered  from  two  aspects.  First,  from  the  aspect  of 
total  inventory  including  all  assets,  and  second,  from  the  aspect  of 
inventory  excluding  major  assets. 
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The  first,  and  overriding,  definition  of  inventory  was  the  dollar 
amount  of  investment  in  all  assets  (aircraft,  buildings,  facilities, 
etc),  support  equipment,  spares  inventory  on  hand,  and  materials  the  FEG 
uses  to  produce  the  defined  throughput.  This  encompassed  all  investment 
in  the  FEG. 

However,  as  the  investment  in  aircraft,  buildings,  and  major 
support  equipment  was  relatively  "fixed",  and  not  in  the  control  of  the 
FEG  Commander,  a  second,  reduced,  definition  of  inventory  was  proposed 
to  provide  a  view  of  performance  trends  of  variables  over  which  the  FEG 
Commander  had  some  control.  The  reduced  definition  was  therefore  the 
dollar  amount  invested  in  spares,  equipment  (excluding  large  capital 
support  equipment  —  which  needs  to  be  operationally  defined),  and  raw 
materials  required  to  produce  the  defined  throughput.  The  reason  for 
this  reduced  definition  of  inventory  was  to  provide  a  measure  which 
gives  a  more  obvious  indication  of  trends  in  terms  of  change  in  the 
"variable  inventory",  as  opposed  to  changes  in  the  larger,  more  static, 
baseline  or  "fixed"  inventory.  Both  definitions  would  provide  useful 
insights  for  the  FEG  Commander,  but  would  be  used  in  different 
situations.  The  full  definition  could  be  used  for  decisions  involving 
capital  investment,  while  the  reduced  definition  could  be  used  for  day- 
to-day  monitoring  of  FEG  performance. 

Aggregate  values  for  fixed  assets  belonging  to  the  FEG  could  be 
provided  from  the  Defence  Asset  Register,  while  aggregate  values  for 
weapons,  support  equipment  and  inventory  items  for  the  FEG  could  be 
provided  from  the  supply  computer  system  being  developed  in  the  Supply 
System  Redevelopment  Project  (SSRP). 

Operating  expense  was  defined  as  all  the  money  the  FEG  spent  in 
turning  the  inventory  into  throughput.  This  would  include  labor  costs, 
cost  of  utilities,  consumption  of  consumables  and  spares,  depreciation 
and  other  "overhead  expenses"  related  to  operating  the  FEG.  The 
"consumption"  of  reparable  spares  would  be  operationally  defined  in 
terms  of  the  cost  of  the  materials,  and  the  services  obtained  from 
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"outside"  the  FEG,  required  to  convert  these  spares  back  to  a 
serviceable  level.  In  practice,  all  these  definitions  would  require 
further  "operationalization"  to  allow  users  to  accurately  understand  the 
components  of  each  definition  and  performance  measure. 

Data  for  the  materiel  consumption  should  be  obtained  from  the 
supply  computer  system  being  developed  under  the  SSRP,  data  for  non¬ 
materiel  goods  and  services  should  be  obtained  from  the  new  financial 
computer  system  being  developed  under  the  Finance  System  Redevelopment 
Project,  and  manpower  and  associated  labor  overhead  data  should  be 
obtained  from  the  new  system  being  developed  in  the  Manpower  System 
Redevelopment  Project. 

The  final  element  of  the  RAAF  management  environment  was  the 
composition  of  the  Strike/Reconnaissance  FEG.  In  broad  logistic  support 
terms,  the  composition  of  the  FEG  was  as  follows; 

a.  Two  Squadrons  of  F-111  aircraft  —  22  Aircraft  tasked  for  strike, 
reconnaissance,  and  training  roles. 

b.  One  Intermediate  Level  Maintenance  Squadron  —  Tasked  primarily  to 
provide  repair  support  at  the  LRU  level. 

c.  One  Depot  Level  Maintenance  Squadron  —  Tasked  primarily  to 
provide  repair  support  at  the  SRU  level,  and  to  provide  depot 
overhaul  of  major  F-111  aircraft  systems. 

d.  One  Supply  Squadron  —  Tasked  to  provide  for  the  storage  and  issue 
of  spares  and  consumable  items  to  the  two  flying  sc[uadrons,  and  to 
the  two  maintenance  squadrons. 

e.  One  Logistics  Management  Squadron  —  Tasked  to  provide  the 
engineering  management,  configuration  management,  reliability 
management,  repair  pipeline  management,  spares  management,  and 
aircraft  repair  and  modification  management  for  the  F-111 
aircraft. 
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Psrformanc*  MBasursiMnt  at  FEG  Laval 

To  the  above  preliminary  operational  definitions  for  the  goal, 
throughput,  inventory  and  operating  expense,  the  following  definitions 
need  to  be  added: 

a.  Scheduled  flying  hours  were  those  hours  planned  on  a  daily/weekly 
basis. 

b.  Budgeted  Flying  hours  were  those  annual  flying  hours  allocated  by 
Higher  Defence  Authority  in  fulfillment  of  FEG  role. 

From  these  definitions,  the  proposed  FEG  performance  measurements 
were  then  defined. 

Effactivanass.  Effectiveness  has  been  defined  as  the  ability  of 
the  organization  to  produce  the  quantity  and  quality  of  output  the 
environment  demands,  and  effectiveness  measures  should  relate  directly 
to  the  output  consumed  by  the  organization's  customers  (14:36-37).  In 
the  case  of  the  Strike/Reconnaissance  FEG,  the  output  consumed  would  be 
"effective  hours”.  This  output  also  provided  the  foundation  for 
developing  the  definition  of  throughput  for  the  FEG.  Maximum 
effectiveness  would  be  achieved  when  the  FEG  was  able  to  launch  all 
missions  on  time,  and  when  the  aircrew  were  able  to  fulfill  all  their 
mission  requirements  during  the  mission  flight. 

Proposed  Measures  of  Effectiveness.  The  proposed  performance 
measures  for  the  effectiveness  of  the  FEG  were  as  follows: 

a.  The  number  of  scheduled  flying  hours  for  missions  actually  started 
on  time,  and  completed  as  "effective  hours",  as  a  percentage  of 
total  scheduled  flying  hours. 

b.  The  number  of  "effective  hours”  completed  as  a  percentage  of 
planned/budgeted  flying  hours. 

c.  The  number  of  "effective  hours”  completed  as  a  percentage  of 
actual  flying  hours  flown. 

The  FBGs  Commander's  aim  would  be  to  have  the  above  derived 
percentages  exhibit  an  increasing  trend,  reflecting  an  improvement  in 
performance.  If  the  percentage  was  already  at  an  acceptable  level,  then 


the  Commander  would  monitor  the  associated  SPC  chart  to  ensure  the 
"process  of  generating  effective  hours"  remained  in  control. 
Alternatively,  if  the  PEG  is  always  obtaining  100%  effective  hours,  a 
new  definition  for  throughput  could  be  developed  which  promoted  other 
personnel  behaviors  which  contributed  to  the  achievement  of  the  PEG'S 
goals.  Note  also  that  the  non-achievement  of  an  effective  hour  equates 
to  the  loss  of  throughput  for  the  PEG. 

Each  of  the  performance  measures  described  above  would  provide 
different  information  on  the  effectiveness  of  the  PEG  Commander  in 
achieving  his  objectives.  The  first  measure  would  provide  an  indication 
of  how  the  PEG  meets  its  day-to-day  flying  objectives,  showing  the  PEG'S 
"readiness"  to  meet  scheduled  taskings.  The  second  measure  would 
provide  an  indication  of  how  effectively  the  PEG  transformed  its 
budgeted  flying  hours  into  "effective  hours".  The  third  measure  would 
provide  a  further  indication  of  the  PEG'S  capability  to  maximize  the 
number  of  "effective  hours"  per  actual  hour  flown. 

These  measures  would  be  computed  on  a  weekly  basis,  with  an  SPC 
chart  being  used  to  track  whether  the  overall  process  was  in  statistical 
control,  and  to  identify  activities  which  were  forcing  the  logistics 
support  process  out  of  control.  The  SPC  chart  would  also  provide 
evidence  of  improvements  in  performance,  or  otherwise,  resulting  from 
decisions  by  PEG  management  to  improve  the  overall  PEG  process. 
Cumulative  figures  for  each  of  these  measures  could  also  be  maintained 
to  provide  evidence  of  the  PEG'S  overall  performance  to  date  during  the 
budget  year. 

In  addition  to  these  overall  PEG  measures,  the  following  examples 
of  global  measures  for  the  components  of  the  PEG  were  also  proposed: 

a.  Por  Aircrew:  (Scheduled  Plying  Hours  minus  Plying  Hours  missed  due 
to  aircrew  problems  [lack  of  crew]}  divided  by  Scheduled  Plying 
Hours . 

b.  For  the  Plightline:  (Scheduled  Plying  Hours  -  Plying  Hours  missed 
or  delayed  because  of  lack  of  aircraft ) /Scheduled  Flying  Hours 
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c.  For  the  Intermediate  Level  Maintenance  Squadron:  (Scheduled  Flying 
Hours  -  Flying  Hours  missed  or  delayed  through  lack  of  an 

LRU) /Scheduled  Flying  Hours. 

d.  For  the  Depot  Level  Maintenance  Squadron:  (Scheduled  Flying  Hours 
-  Flying  Hours  missed  or  delayed  through  lack  of  an  SRU,  or 
through  aircraft  delayed  in  depot  level  maintenance) /Scheduled 
Flying  Hours. 

e.  For  the  Supply  Support  Squadron:  (Scheduled  Flying  Hours  -  Flying 
Hours  missed  or  delayed  through  lack  of  any  reparable  item,  or 
consumable  stores  item) /Scheduled  Flying  Hours. 

f.  For  the  Logistics  Management  Squadron:  (Scheduled  Flying  Hours  - 
Flying  Hours  missed  or  delayed  through  the  lack  of  any  repayable 
or  consumable  stores  item,  or  mission  aborted) /Scheduled  Flying 
Hours. 

Each  of  these  component  measures  would  provide  an  indication  of 
the  component's  performance  in  attaining  the  FEG  goals.  For  example, 
the  measure  for  the  Flightline  would  indicate  the  ability  of  the 
Flightline  to  meet  the  FEG  goals,  and  the  measure  for  the  Logistics 
Management  Squadron  would  reflect  on  a  combination  of  the  reliability  of 
aircraft  components  and  the  effectiveness  of  the  engineering  management, 
repair  pipeline  management  and  spares  management  functions  performed  by 
that  Logistics  Management  Squadron. 

The  measures  would  rely  on  the  identification  of  the  primary  cause 
of  a  failure  to  accomplish  an  "effective  hour”.  The  delay  or  loss  of 
throughput  would  be  a  significant  event  in  preventing  the  FEG  from 
meeting  its  goal,  and  therefore  would  warrant  immediate  identification 
of  the  primary  cause  of  the  delay  or  loss.  Once  identified,  this  cause 
of  delay  or  loss  of  effective  hours  would  be  used  to  allocate  the  loss 
of  effective  hours  to  the  appropriate  logistic  support  elements  which 
"participated"  in  creating  the  loss. 

However,  these  performance  measures  were  not  designed  to  apportion 
blame,  even  though  the  cause  of  any  problems  might  be  "down  the  line". 
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Instead,  the  measures  were  designed  to  instill  more  communication 
between  all  elements  making  up  the  FEG.  In  many  cases,  all  areas  will 
share  some  of  the  blame  for  "failures  to  fly  on  time". 

For  example,  if  a  flight  was  delayed  due  to  the  unavailability  of 
an  LRU,  then  the  FEG's  components'  individual  performances  would  be 
affected  for  the  following  reasons: 

a.  The  Flightline  —  the  scheduled  flight  did  not  depart  on  schedule. 

b.  The  Supply  Support  Squadron  —  the  Flight line  was  not  provided 
with  the  required  LRU. 

c.  The  Intermediate  Level  Maintenance  Squadron  —  the  required  LRU 
was  not  available  for  issue  into  the  Supply  Support  Squadron  at 
the  required  time. 

d.  The  Logistics  Management  Squadron  —  the  LRU  pipeline  failed  to 
provide  sufficient  serviceable  LRUs  to  allow  the  scheduled  flying 
hours . 

Having  been  "saddled"  with  a  "failure  to  perform",  each  area  would 
have  a  basis  for  seeking  out  the  causes  for  their  "poor"  performance. 
However,  detailed  investigations  of  these  losses  would  only  need  to  be 
undertaken  when  the  SPC  chart  indicated  that  the  process  of  generating 
effective  hours  was  out  of  statistical  control.  Correction  of  out  of 
control  performance  would  require  the  logistics  support  components  to 
liaise  with  the  other  components  of  the  FEG  to  improve  the  coordination 
of  requirements  to  meet  future  scheduled  flying  hour  commitments. 

Mote  was  made  that  the  performance  indicators  would  not  be 
comparable  between  FEG  components,  e.g.  the  Flightline  would  always  have 
a  "poorer”  performance  rating  than  the  maintenance  depot  because  the 
Flightline  was  at  the  top  of  the  support  pyramid.  The  important  point 
here  was  to  measure  relative  changes  in  component  performance  rather 
than  measure  absolute  levels  of  component  performance.  These  relative 
changes  in  performance  would  then  be  used  to  indicate  whether  the 
support  component's  performance  was  exhibiting  continuous  improvement. 
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Using  the  proposed  performance  measurement  approach,  individual 
FEG  components  would  have  the  incentive  to  improve  their  performance, 
within  their  total  operating  environment,  as  a  means  to  improving  their 
effectiveness  rating.  One  avenue  for  improving  performance  would  be  for 
a  support  component  to  liaise  with  the  other  support  components  to 
improve  logistics  processes  such  that  all  components  see  an  increase  in 
individual  component  effectiveness,  as  well  as  an  increase  FEG 
effectiveness  as  a  whole. 

A  hypothetical  example  of  how  these  measures  would  be  used  was 
provided  at  Appendix  D. 

Efficiency.  Efficiency  is  generally  defined  in  terms  of  the 
ratio  of  output  to  inputs.  Two  types  of  efficiency  measures  would  be 
used  for  the  Strike/Recom..  .ssance  FEG: 

a.  effective  hours  per  dollar  of  operating  expense  (or  for  reporting 
purposes,  dollars  of  operating  expense  per  effective  flying  hour); 
and 

b.  effective  hours  per  dollar  of  inventory  investment  (or  dollars 
invested  in  inventory  per  effective  hour). 

In  general  terms,  the  FEG  Commander's  aim  would  be  to  see  the 
dollars  of  operating  expense  per  effective  hour,  and  the  dollars  of 
inventory  per  effective  hour,  exhibit  a  declining  trend.  This  would 
indicate  a  a  improvement  in  the  efficiency  of  FEG  operations. 

Proposed  Efficiency  Maasuces.  Based  on  the  foregoing  discussion, 
exeunples  of  the  proposed  efficiency  measures  were: 

a.  For  the  FEG: 

(1)  The  operating  expense  for  the  whole  FEG  per  effective  hour. 

(2)  The  inventory  investment  for  the  whole  FEG  per  effective 
hour. 

b.  For  the  Flightline: 

(1)  The  operating  expense  for  the  Flightline  per  effective  hour. 

(2)  The  inventory  investment  for  the  Flightline  per  effective 
hour. 
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c.  For  the  maintenance  squadrons: 

(1)  The  operating  expense  for  the  maintenance  squadron  per 
effective  hour. 

(2)  The  inventory  investment  for  the  maintenance  squadron  per 
effective  hour. 

d.  For  the  Supply  Support  Squadron: 

(1)  The  operating  expense  for  the  Supply  Support  Squadron  per 
effective  hour. 

(2)  The  inventory  investment  for  the  Supply  Support  Squadron  per 
effective  hour. 

With  the  use  of  appropriate  "cost  records"  obtained  from  the  FEG 
Commander's  financial  controller,  managers  of  FEG  support  components 
should  be  able  to  identify  reasons  for  any  change  in  efficiency,  and 
should  be  able  to  determine  what  action,  if  any,  would  be  required  to 
correct  any  decreases  in  efficiency.  Note  that  the  efficiency  measures 
used  for  FEG  components  are  relative  measures,  and  should  be  used  to 
assess  trends  in  efficiency  improvement  or  decline. 

The  measures  proposed  in  this  research  were  "global”  measures  for 
use  by  the  FEG  Commander,  as  they  related  principally  to  an  assessment 
of  the  performance  of  the  FEG  as  a  whole.  Other  "internal"  performance 
measures  should  be  developed  for  each  organization  within  the  FEG.  Each 
of  these  internal  performance  measures  needs  to  be  constructed  so  that 
it  "dovetails"  into  the  global  measures.  That  is,  these  internal 
measures  should  be  designed  in  such  a  way  that  they  encourage  personnel 
to  use  behaviors  which  support  the  goals  of  the  entire  FEG.  The 
development  of  these  internal  measures  was  beyond  the  scope  of  this 
research. 

Cenparlsen  of  Perfozswnee  Measures  to  IMB  Ouidelines 

A  comparison  of  the  performance  measurements,  derived  in  the 
previous  section  to  the  guidelines  determined  in  answer  to  Investigative 
Question  1  was  undertaken  to  answer  Investigative  Question  4. 


Xnvttstigativ*  Qu*stion  4.  Does  the  Strike/Reconnaissance  PEG 
example  satisfy  the  performance  measurement  requirements  of  the 
PMB  process? 

An  assessment  of  the  developed  performance  measures  against  each 
of  the  guidelines  determined  in  answer  to  Investigative  Question  1  was 
provided  in  the  following  paragraphs. 

Assessment  of  Effectiveness  of  Programs  Against  Objectives.  The 
proposed  measures  provided  a  surrogate  measurement  of  performance  for 
use  in  assessing  RAAF  goals  la(iv),  lb,  2e  and  3a,  as  listed  in  Appendix 
C.  The  effectiveness  measures  provided  would  show  the  number  of 
"effective  hours"  achieved  against  scheduled  operations,  against 
budgeted  hours,  and  against  actual  hours  flown.  These  measures  provide 
an  indication  of  the  PEG'S  readiness,  and  of  the  PEG'S  capability  to 
meet  the  operational  targets  set  by  the  CDP.  While  the  measures  do  not 
provide  an  exact  measurement  of  readiness,  the  measures  as  designed 
encourage  organizational  behavior  to  be  directed  towards  meeting  the 
objectives  of  the  RAAP  program  under  PMB. 

Efficiency  of  Resource  Use.  The  measures  as  proposed  did  not 
provide  a  measure  of  absolute  efficiency  of  the  PEG.  However,  they  did 
provide  the  basis  for  assessing  trends  in  efficiency.  Tracking  of  these 
trends  would  allow  the  PEG  Commander  to  determine  the  relative 
efficiency  of  the  PEG  against  its  past  performance.  This  type  of 
measurement  was  considered  to  be  ideal  for  an  organization  which  was 
pursuing  a  policy  of  continual  process  improvement.  As  the  RAAP  had 
adopted  RAAP  Quality  (RAAPQ)  as  its  version  of  Total  Quality  Management, 
the  proposed  efficiency  measures  would  provide  the  PEG  Commander  with 
the  tools  to  monitor  continuous  cost  improvement  under  RAAPQ. 

Perfozsumce  in  Financial  and  Mon-Financial  Tersui.  The  proposed 
measures  were  couched  in  a  combination  of  financial  and  non-f inancial 
variables.  The  proposed  measures  were  therefore  in  accord  with  the  PMB 
process. 
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Ou^coM  Orientation.  By  defining  the  required  outcome  of  the  FEG 
activities  to  be  the  attainment  of  effective  hours,  the  concentration  of 
the  measures  on  "effective  hours”  provided  the  required  orientation  to 
measuring  the  "outcomes"  of  the  FEG's  efforts. 

Perfozauuice  in  Terms  of  FEGs.  The  measures  all  relate  to  the 
performance  of  the  FEG.  Some  of  measures  relate  to  the  whole  FEG 
directly,  e.g  effective  hours  per  scheduled  flying  hour  for  the  whole 
FEG,  while  the  remainder  measure  FEG  logistics  support  component 
performance  in  terms  of  total  FEG  performance,  e.g.  the  operating 
expense  per  "effective  hour"  for  the  Supply  Support  Squadron. 

Indicators  May  Differ  Between  FEGs.  The  indicators  were 
developed  specifically  for  the  Strike/Reconnaissance  FEG.  The  same  set 
of  indicators  may  not  be  directly  applicable  to,  for  example,  the 
Strategic  Transport  Group,  since  the  definition  of  throughput  would 
probably  be  different  for  the  strategic  transport  activity.  However, 
with  an  appropriate  definition  of  throughput,  similar  performance 
measures  could  be  developed  using  the  concepts  adopted  in  this  research. 
This  difference  between  performance  measures  for  different  FEGs  would  be 
acceptable  under  the  PMB  guidelines. 

Different  Measures  at  Different  X,evels.  The  indicators  were  not 
uniform  across  components  within  the  FEG  owing  to  the  hierarchical 
nature  of  the  measures.  This  should  not  provide  a  problem  as  each 
component  being  assessed  is  performing  against  itself.  The  aim  of  each 
component  should  be  to  continually  improve  its  own  performance  through 
appropriate  interaction  with  other  support  elements.  This  difference  in 
measures  between  support  components  was  permitted  under  the  PMB 
guidelines. 

Emphasis  on  Program  and  Evaluation.  The  whole  focus  of  the 
proposed  performance  measures  was  on  the  outcomes  produced  by  the 
Strike/  Reconnaissance  FEG.  The  proposed  measures  would  therefore 
provide  the  vehicle  through  which  the  FEG  outcomes  could  be  evaluated. 
Thus,  the  proposed  measures  would  be  in  accord  with  the  PMB  guidelines. 
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Concantration  on  Outcomaa  .lather  than  on  Eapanditura  Achiavaaiant . 
Targets  for  the  proposed  performance  measures  were  directed  at  producing 
the  required  output  at  the  minimum  cost.  The  emphasis  on  trends  which 
show  continuous  improvement  in  performance  at  a  decreasing  cost  could 
mean  that  the  FEG  will  produce  its  output  at  a  cost  lower  than  budgeted 
cost.  This  emphasis  on  improvement  and  cost  reduction  does  not  allow  a 
mentality  of  expenditure  achievement,  as  expenditure  just  for  the  sake 
of  expenditure  achievement  will  result  in  either  stagnant  or  poorer 
efficiency  performances.  Thus,  the  proposed  measures  support  the 
concentration  on  outcomes  rather  than  supporting  a  concentration  on 
expenditure  achievement. 

Mo  Burden  on  Combat  Units.  The  collation  of  operating  expense 
and  inventory  investment  data  for  the  proposed  efficiency  performance 
measures  could  be  undertaken  by  the  FEG  Commander's  financial  controller 
through  on-line  access  to  the  databases  provided  the  SSRP,  MSRP,  and 
FSRP  systems.  Thus,  no  actual  collation  of  data  needs  to  be  undertaken 
by  the  components  of  the  FEG  except  for  the  recording  "effective  hours” 
against  scheduled  flying  hours,  and  the  recording  of  reasons  for  the 
inability  to  achieve  scheduled  flying  hours  —  the  data  required  for  the 
effectiveness  performance  measure.  The  first  of  these  two  tasks  for  the 
FEG  components  would  not  be  particularly  onerous,  while  the  second  task 
would  be  a  necessary  process  which  should  be  undertaken  if  process 
improvement  was  sought.  The  proposed  performance  measures  would  require 
that  there  be  more  communication  between  the  flying  squadrons  and  all 
their  logistic  support  elements  to  allow  these  support  elements  to  be 
more  attuned  to  the  daily  scheduling  needs  of  the  flying  squadrons. 

This  communication  would  be  a  necessary  condition  is  a  necessary 
condition  which  needs  to  be  fulfilled  if  the  whole  process  of  producing 
effective  hours  is  to  show  continual  improvement.  Thus,  the  proposed 
performance  measures  provide  no  unnecessary  new  burdens  for  the  combat 
units. 
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FEGs  to  Participat*  in  tha  Saif -Evaluation  of  Parf ozmanca .  The 
use  of  the  proposed  performance  measures  would  provide  a  valuable  tool 
for  allowing  the  PEG  Commander  to  evaluate  his  PEG'S  performance.  The 
performance  measures  would  form  the  basis  which  would  allow  the  PEG 
Commander  to  identify  when  problems  have  occurred,  and  to  identify  the 
possible  source  of  problems.  By  measuring  the  performance  of  the  PEG, 
the  PEG  Commander  would  be  able  to  participate  in  the  evaluation  of  his 
PEG'S  performance,  thus  meeting  the  PMB  guidelines. 

Based  on  the  above  comparison  of  the  proposed  performance  measures 
with  the  PMB  guidelines  derived  in  answer  to  Investigative  Question  1, 
the  proposed  performance  measures  were  within  the  guidelines  specified 
by  the  Australian  DOO  PMB  process 

Limitations  of  Proposed  Performance  Measurements 

Being  a  first  attempt  to  use  TOC  measures  for  assessing  PEG 
performance,  the  proposed  measures  may  need  some  further  refinement. 
Identifiable  limitations  of  the  measures  include: 

a.  The  measures  did  not  address  sustainability  performance.  These 
would  have  to  be  couched  in  terms  of  the  rates  of  effort  detailed 
in  the  classified  document  01 (AP)  ASD208. 

b.  The  emphasis  of  the  measures  was  on  the  effectiveness  and 
efficiency  of  logistics  support.  No  attempt  was  made  to  factor  in 
other  support,  such  as  medical  and  general  administrative  support. 
This  could  be  achieved  at  PEG  level,  through  modifications  to  the 
definitions  of  inventory  and  operating  expense,  to  include  the 
investment  and  operating  expense  associated  with  these  facilities. 

c.  The  research  did  not  use  the  definitions  of  operational  readiness 
as  provided  in  CORO  owing  to  classified  nature  of  the  document, 
and  the  lack  of  access  to  document  at  the  remote  location  of  the 
researcher  (remote  in  terms  of  proximity  to  HQAOP). 

d.  The  paper  did  not  provide  a  treatment  of  how  depreciation  of 
military  equipment  should  be  handled. 
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e.  The  performance  measures  developed  were  surrogate  measures. 
Therefore,  managers  must  be  careful  in  their  analysis  of  the 
measures. 

f.  Following  on  from  sub-para  e,  the  definition  of  effective  hours 
was  provided  by  the  researcher.  The  acceptance  of  this  definition 
would  require  input  from  senior  RAAF  managers  as  to  their 
perceptions  of  "effective  hours". 

g.  An  underlying  assumption  in  these  performance  measures  was  that  an 
aircraft  behaved  in  a  serial  fashion,  that  is,  all  aircraft 
components  required  for  a  scheduled  mission  are  equally  necessary 
for  the  mission  to  be  accomplished.  This  was  a  valid  assumption 
in  terms  of  treating  the  aircraft  as  being  configured  to  perform  a 
specific  mission  —  all  systems  on  the  aircraft  may  not  be  100% 
serviceable;  however,  all  systems  required  to  perform  the  mission 
need  to  be  serviceable. 

h.  Part  of  the  operating  expense  of  the  PEG  would  be  services 
consumed  which  were  provided  by  elements  of  the  RAAF  outside  the 
PEG.  These  services  would  not  have  an  explicit  cost  similar  to 
those  costs  for  services  provided  by  outside  vendors.  Therefore, 
an  internal  transfer  cost  would  need  to  be  applied  to  those 
services.  This  research  did  not  addressed  how  this  internal 
transfer  cost  should  be  computed. 

i.  The  need  for  an  accrual  accounting  approach  for  the  consumption  of 
externally  provided  services,  e.g.  monthly  utility  payments,  was 
not  been  addressed  in  the  research. 

Despite  these  limitations,  the  measures  as  proposed  do  provide  the 

PEG  Commander  with  an  indication  of  the  relative  PEG  performance,  over 

time,  against  RAAP  goals. 

Susnary 

Chapter  IV  provided  answers  for  Investigative  Questions  1  to  4. 

In  answer  to  Investigative  Question  1,  a  review  of  the  Australian 
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Defence  Department  Program  Management  and  Budgeting  System  provided  12 
guidelines  which  should  be  used  in  developing  a  performance  measurement 
system  for  a  Force  Element  Group  in  the  Australian  Defence  Force.  These 
guidelines  were  later  used  to  evaluate  the  set  of  performance  measures 
developed  for  the  Strike/Reconnaissance  Force  Element  Group. 

The  answer  to  Investigative  Question  2  involved  the  exeunination  of 
some  of  the  principles  of  the  Theory  of  Constraints.  Relevant 
principles  relating  to  the  measures  of  throughput,  inventory  operating 
expenses  were  addressed,  and  these  principles  were  adapted  for  use  in  a 
not-for-profit  organization. 

Performance  measures  for  the  Strike/Reconnaissance  FEG  were 
developed  in  answer  to  investigative  Question  3.  The  performance 
measures  were  surrogate  measures  designed  to  provide  an  indication  of 
the  PEG'S  attainment  of  the  RAAF  goals  la(iv),  lb,  2e,  and  3a  set  out  in 
Appendix  C.  The  performance  measures  were  developed  as  global  measures 
for  the  use  of  the  FEG  Commander,  and  each  measure  developed  could  be 
classified  as  one  of  two  types  of  measure,  either  a  measure  of 
effectiveness  or  a  measure  of  efficiency.  The  former  type  of  measure 
was  designed  to  provide  an  indication  of  how  well  the  FEG  met  the  RAAF's 
goals,  while  the  latter  type  of  measure  provided  an  indication  of 
resource  utilization  whilst  achieving  those  goals.  Both  sets  of 
performance  measures  were  designed  to  show  trends  in  performance,  rather 
than  provide  absolute  measures.  This  approach  allows  the  FEG  Commander 
to  assess  whether  improvements  in  performance  are  in  fact  being  obtained 
under  the  RAAFQ  philosophy  of  continuous  improvement. 

The  answer  to  Investigative  Question  4  was  achieved  by  comparing 
the  attributes  of  the  performance  measurements  developed  in  answer  to 
Investigative  Question  3  to  the  guidelines  provide  from  the  answer  to 
Investigative  Question  1.  The  comparison  showed  that  the  developed 
performance  measures  fell  within  the  guidelines  obtained  form  the  review 
of  the  PMB  Reference  Manual. 
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The  chapter  concluded  with  an  examination  of  the  potential 
limitations  of  the  measures  which  were  developed.  These  potential 
limitations  provide  fertile  ground  for  further  investigations  into  the 
performance  measures  for  the  RAAF. 
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V.  Conclusions  and  Rseonmsndations 

Overview 

The  purpose  of  this  research  was  to  examine  the  use  of  the 
performance  measurement  concepts  adopted  in  the  Theory  of  Constraints, 
and  assess  the  applicability  of  their  use  as  a  performance  measurement 
methodology  for  the  RAAF  to  monitor  the  accomplishment  of  its 
organizational  goals.  The  research  was  undertaken  as  a  response  to  the 
requirements  of  the  Department  of  Defence  Program  Management  and 
Budgeting  initiatives  for  an  "outcomes  based”  performance  measurement 
system,  and  as  response  to  the  requirement  of  the  DLSPG  to  develop 
appropriate  performance  indicators. 

The  research  process  began  with  a  review  of  traditional  cost 
accounting  based  performance  measures.  As  presented,  the  review  showed 
that  these  traditional  measures  were  no  longer  relevant  in  today's 
environment  where  organizations  were  striving  for  continuous  process 
improvement.  In  many  cases,  these  traditional  measures  were  seen  to  be 
a  hindrance  to  organizations  seeking  improvement  in  their  organizational 
processes. 

Many  of  those  authors  criticizing  the  traditional  measures 
suggested  Activity  Based  Costing  (ABC)  as  an  alternative.  The 
literature  review  examined  the  applicability  of  using  ABC  systems  to 
provide  performance  measures.  The  examination  found  that,  while  ABC 
provided  a  better  basis  for  accounting  and  reporting  on  past 
performance,  it  was  still  not  responsive  enough  to  provide  information 
to  monitor  the  day-to-day  performance  of  the  organization.  However,  ABC 
provided  a  good  basis  for  semi-annual  or  annual  reporting  purposes  to 
allow  cost  comparisons  with  other  organizations. 

The  literature  review  showed  that  the  current  trend  for 
performance  measures  was  the  adoption  of  measures  which  were  based  on 
both  financial  and  non-f inancial  data.  This  was  particularly  so  for 


government  organizations  which  were  confronted  with  unique  problems  when 
measuring  organizational  performance.  Unlike  for-profit  organizations, 
many  government  organizations  were  unable  to  use  the  concepts  of  profit 
and  loss  as  a  measure  of  effectiveness  and  efficiency.  Instead,  many 
government  organizations  had  to  find  alternative  definitions  of  what 
constituted  satisfactory  organizational  outcomes.  The  intangible, 
service  oriented  nature  of  the  services  of  these  organizations  meant 
that  surrogate  or  proxy  measures  of  performance  needed  to  be  derived  to 
allow  performance  assessment  of  these  organizations.  The  literature 
review  concluded  that  much  research  and  experimentation  needs  to  be 
undertaken  to  develop  performance  measures  for  government  organizations. 

To  examine  whether  the  concepts  of  the  Theory  of  Constraints  were 
applicable  to  the  RAAF,  four  investigative  questions  were  addressed. 

The  answers  to  these  investigative  questions  provided  the  basis  for  the 
following  conclusions  and  recommendations. 

Conclusions 

As  a  result  of  the  research,  five  conclusions  were  drawn.  First, 
the  development  of  performance  measures  in  the  government  environment 
was  not  a  trivial  task.  Difficulties  arose  when  trying  to  operationally 
define  the  intangible  outputs  of  the  organization  such  that  these 
outcomes  could  be  adequately  measured.  In  many  cases,  managers  had  to 
resort  to  the  use  of  proxy  measures.  However,  the  use  of  proxy  measures 
must  be  approached  with  much  care  since  the  measure  defined  to  represent 
performance  would  result  in  driving  behavior  towards  the  achievement  of 
performance  in  terms  of  the  proxy  outcome.  The  objectives  of  these 
measures  must  be  to  provide  performance  measurements  that  not  only 
reflect  the  goal  of  the  organization,  but  also  provide  the  incentive  for 
personnel  to  modify  their  behavior  to  support  the  organizational 
goal(s) . 

Second,  PMB  allowed  a  large  degree  of  flexibility  in  the  selection 
of  performance  measures.  The  main  criterium  was  that  the  measures 
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developed  had  to  provide  a  reasonable  assessment  of  the  outcomes  of 
activities  compared  to  the  objectives  of  the  Department  of  Defence. 

Third,  the  concepts  of  the  Theory  of  Constraints  provided  a 
possible  foundation  for  developing  performance  measures  for  the  RAAF. 

The  TOC  concepts  placed  a  heavy  emphasis  on  defining  the  goals  of  the 
organization,  and  then  developing  performance  measures  which  indicated 
the  achievement  of  those  goals.  The  use  of  such  measures,  combined  with 
the  analysis  of  statistical  process  control  charts  based  on  these 
measures,  provide  a  vehicle  for  the  adoption  and  the  monitoring  of  a 
RAAF  process  of  continual  improvement. 

Fourth,  the  definition  of  performance  measurements  for  the  RAAF 
will  be  an  ongoing  task.  Experimentation  with  definitions  for  surrogate 
measures  will  have  to  be  undertaken  until  the  appropriate  measures  are 
found  that  result  in  the  promotion  of  personnel  behavior  directed  at 
achieving  the  objectives  of  the  RAAF.  These  definitions  will  need  to  be 
refined,  initially  through  management  discussion,  then  through 
experience  from  using  the  new  definitions.  The  success  of  any 
performance  measure  for  the  RAAF  will  be  heavily  dependent  on  the 
selection  of  appropriate  measurable  definitions  of  outcomes  or 
throughput . 

Finally,  the  definitions  proposed  in  the  case  study  were  not 
considered  to  be  perfect,  and  throughput  for  the  Strike/Reconnaissance 
FEG  may  need  to  be  more  precisely  defined.  However,  the  measures  do 
provide  a  starting  point  for  development  of  RAAF  FEG  performance 
measures.  The  concepts  presented  by  TOC  provide  a  useful  framework  for 
the  development  of  performance  measures  which  reflect  the  goals  of  the 
FEG  and  the  RAAF. 

Recoanendations 

This  research  has  established  that,  with  the  appropriate 
definition  of  throughput,  the  performance  measurement  concepts  of  the 
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Theory  of  Constraints  can  be  applied  to  the  Force  Element  Groups  of  the 

RAAF.  The  recommendations  following  from  this  research  are: 

a.  for  the  RAAF  to  consider  the  adoption  of  the  performance 
measurement  concepts  of  the  Theory  of  Constraints  as  a  basis  for 
developing  performance  measures  for  use  by  the  RAAF  Force  Element 
Group  Commanders; 

b.  for  the  RAAF  Force  Element  Group  Commanders  to  identify 
appropriate  definitions  of  throughput  for  their  FEGs  — 
definitions  which  reflect  the  goals  of  their  individual  Groups  in 
terms  of  meeting  overall  RAAF  goals;  and 

c.  for  the  project  leaders  of  the  Supply,  Manpower,  and  Finance 
System  Redevelopment  Projects  to  ensure  that  FEG  Commanders  have 
on-line  access  to  data  which  allows  the  FEG  Commander  to  easily 
compute  his  performance  measurements. 

In  addition  to  these  recommendations,  five  areas  have  been 

identified  for  further  research: 

a.  the  development  of  the  concept  of  depreciation  in  a  military 
environment; 

b.  the  development  of  a  basis  for  establishing  transfer  prices  in  a 
military  environment; 

c.  the  development  of  simple  techniques  to  account  for  the 
consumption  of  resources  other  than  those  resources  accounted  for 
in  the  RAAF  supply  system  e.g.  the  consumption  of  utilities; 

d.  the  investigation  of  the  use  of  the  Theory  of  Constraints 
performance  measurement  concepts  to  develop  internal  performance 
measurements  for  the  FEG  sub-components;  and 

e.  the  comparison  of  the  Theory  of  Constraints  concept  of  performance 
measurement  with  alternative  performance  measurement  concepts  such 
as  provided  in  the  Computer-Aided  Manufacturing-International 
Project  and  the  GASB  Service  Effects  and  Accomplishments  Project. 
In  closing,  it  is  the  researcher's  studied  opinion  that  the 

concepts  of  the  Theory  of  Constraints  can  be  applied  to  the  Force 
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Element  Groups  of  the  RAAF.  However,  considerable  attention  must  be 
given  to  defining  throughput  such  that  definition  provides  a  reasonable 
measure  of  the  outcomes  of  activities  in  terms  of  the  goals  and 
objectives  of  the  RAAF. 
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J^pandlx  A:  D«f«nc*  Logistics  Strat*gic  Planning  Quid* 
Objaetlvas  and  Stratagiaa 


Objaetlv*  I:  Express  logistics  requirements  and  capabilities  in 
readiness  and  sustainability  terms. 

Enabling  ^jective  1.2:  Develop  processes  to  determine  logistics 
requirements  from  operational  needs  and  express  logistics  capability  in 
operational  terms. 

Strategy  Z.2.3:  Support  logistics  decision  making  processes  by 

developing  methodologies  and  models  which: 

a.  tie  logistics  requirements  computations  to  preparedness 
objectives; 

b.  express  the  levels  of  logistics  performance  achievable  from 
given  resources  in  operationally  meaningful  terms;  and 

c.  allow  effective  budgeting,  resource  allocation  and  risk 
analysis. 

Strategy  1.2.4:  Organize  logistics  data  by  employing  a  logistics 

information  systems  architecture  which  links: 

a.  equipments  to  force  elements; 

b.  hierarchical  relationships  within  equipments; 

c.  logistics  activities  to  specific  equipments  and  force 
elements;  and 

d.  entitlements,  availability  and  consumption  of  resources  to 
specific  activities,  equipments  and  force  elements. 


Objective  III:  Achieve  weapon  system  and  equipment  operational 
availability  targets. 

Enabling  Objective  ZZZ.l:  Emphasize  weapon  system  management 
principles  in  all  aspects  of  logistics. 

Strategy  ZZZ.l. 5:  Establish  organizational  structures, 
performance  assessment  processes  and  systems  that  focus  on  weapons 
systems  availability  and  resource  allocations  against  weapon 
system  priorities. 

Objective  ZV:  Field  integrated  logistics  information  systems  to 
provide  responsive  decision  support  and  improve  logistics  performance. 

Enabling  (E»jectlve  ZV.2:  Progressively  integrate  logistics  information 
systems. 

Strategy  ZV.2. 4:  Interface  logistics  information  systems  with 
operational  and  resource  information  systems  to  facilitate 
performance  measurement  and  resource  allocation. 

Strategy  ZV.2. 5:  Establish  an  information  architecture  which 
enables  logistics  resource  entitlements,  availability  and 
consumption  to  be  linked  to  force  elements,  weapon  systems, 
equipments  and  logistics  activities. 


Objactiv*  VIZ:  Improve  the  quality  and  management  of  logistics 
operations. 

Enabling  Objective  VIZ. 4.1:  Improve  programming,  budgeting  and 
management  of  resources. 

Strategy  VIZ. 4.1:  Ensure  that  logistics  inputs  to  PMB  processes 
are  structured  to  attribute  financial  estimates  and  budgets  to 
preparedness  objectives. 

Strategy  VIZ. 4. 2:  Ensure  that  resources  management  processes  are 
structured  to  attribute  the  costs  of  logistics  activities  directly 
to  preparedness  objectives. 

Strategy  VIZ. 4. 3:  Improve  methods  of  identifying  cost  to  users 
and  explore  means  of  making  users  accountable  for  goods  and 
services  consumed. 

Strategy  VIZ. 4. 4:  Develop  the  capability  to  devolve  resource 
management  authority  to  the  unit  and  work  center  level. 

Strategy  VIZ. 4. 5:  Devolve  financial  commitment  and  expenditure 
authority  to  the  lowest  level  of  responsibility  at  which 
accountability  can  be  effectively  maintained  and  controlled. 

Strategy  VZZ.4.6:  Ensure  that  financial  guidelines  and 
procedures  are  introduced  to  facilitate  the  adoption  of  commercial 
practices. 
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J^pttndix  B:  Ouid«lin«s  for  tho  Dovolopmont  of  Motrics 


The  Metrics  handbook  offers  advice  on  the  attributes  of  a  good 
metric  and  on  the  detail  required  in  the  operational  definition  of  a 
metric.  A  summary  of  the  attributes  and  detail  required  are  given  in 
the  following  paragraphs. 


Attributes  of  a  Good  Metric 

1.  The  metric  is  accepted  as  meaningful  to  the  customer. 

2.  The  metric  tells  how  well  organizational  goals  and  objectives  are 
being  met  through  processes  and  tasks. 

3.  The  metric  should  be  simple,  understandable,  logical  and 
repeatable . 

4.  The  metric  shows  a  trend. 

5.  The  metric  is  unambiuously  defined. 

6.  Data  is  economical  to  collect. 

7.  The  metric  can  be  produced  in  a  timely  fashion. 

8.  The  metric  drives  the  appropriate  action. 


Detail  Required  in  operational  Definition  of  a  Metric 

1.  An  unambiguous  descrition  of  the  metric. 

2.  The  population  that  the  metric  will  include. 

3.  The  frequency  of  the  measurement. 

4.  The  source  of  the  data. 

5.  Any  equations  required  in  doing  the  measurement. 

6.  Precise  definition  of  key  terms. 

7.  A  desription  of  the  graphic  presentation  that  will  eventually  be 
used  to  display  the  data. 

8.  The  customer  of  the  metric. 

9.  the  accountable  process  owner. 

10.  The  desired  outcome  expressed  in  terms  of  positive  or  negative 
trend  (not  a  numerical  goal). 

11.  The  link  between  the  process  being  measured,  the  organization's 
goals,  and  the  higher  command  goals.  (1:3-2) 

Finally,  a  metric  presentation  has  two  parts:  the  metric 
descriptor,  and  the  graphic  presentation  of  the  data  (1:3-2). 
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J^ppandix  C:  Stated  Mission  and  Goals  of  the  RAAF 


Mission:  conduct  effective  strategic  and  tactical  air  operations  as  an 
independent  force,  or  as  part  of  a  joint  or  combined  force  in  the 
pursuit  of  Australia’s  defence  and  national  interests.  (29:5) 


Goal  1:  Preparedness 

a.  Develop  and  maintain  air  forces  capable  of  conducting  and 
sustaining  effective  defensive  and  offensive  air  operations  by: 

(i)  Accurately  assessing  our  capability  shortfalls  and  planning 
to  update  or  acquire  the  required  dLmprovements. 

(ii)  Ensuring  our  air  power  doctrine  tactics  and  procedures 
maximize  our  combat  potential. 

(iii)  Ensuring  that  we  can  operate  effectively  with  the  Navy  and 
the  Army. 

(iv)  Providing  quality  logistics  support  for  all  our  operations. 

b.  Maintain  operational  readiness  by  achieving  the  targets  specified 
by  CDF's  Operational  Readiness  Directive. 

c.  Continue  surveillance  and  intelligence  gathering. 

d.  Maintain  and  develop  airfields,  and  combat  oriented  facilities  and 
infrastructure,  (bases,  facilities,  communications  networks  etc) 
particularly  in  likely  areas  of  operations. 

e.  Develop  the  RAAF  Reserve,  including  the  Ready  Reserve  and  the 
Aircrew  Contingency  Reserve  Scheme,  to  provide  an  operational 
surge  capability. 

f.  Be  able  to  operate  with  our  friends  and  allies  by  participating  in 
international  defence  forums  such  as  the  Air  Standardisation 
Coordination  Committee  -  Australia/USA/  New  Zealand/Canada/UK. 

g.  Develop  and  maintain  operating  and  support  procedures  with  our 
allies  to  enable  rapid  build-up  of  equipment  if  the  need  arises. 


h.  Develop  policies  to  increase  our  level  of  self-sufficiency 
through  active  association  with  Australian  industry. 


Goal  2:  Operational  Excellence 

a.  Strive  for  excellence  in  independent  air  operations. 

b.  Ensure  combat-oriented  training. 

c.  Develop  joint  doctrine  and  procedures  to  ensure  timely  and 
effective  support  for  the  Navy  and  Army. 

d.  Participate  in  Australian  Joint  Service  and  Combined  International 
exercises. 


Ensure  timely  and  effective  maintenance  and  logistic  support. 


Goal  3:  Productivity 

a.  Search  continuously  for  better  ways  to  do  the  job. 

b.  Introduce  better  management  and  work  practices  through  the 
application  of  RAAF  Quality  (RAAFQ) . 

c.  Devolve  responsibility  to  the  lowest  appropriate  level. 

d.  Selectively  incorporate  advanced  technology  systems,  particularly 
in  relation  to  information  systems. 

e.  Ensure  that  our  training  prepares  us  for  our  jobs. 

f.  Be  receptive  to  new  ideas,  however  different  or  novel  they  may 

appear . 

g.  Nurture  technological  excellence  and  innovation 

Goal  4:  Our  People 

a.  Continue  our  commitment  to  our  members  and  their  families. 

b.  Maintain  a  responsive  approach  to  career  management  and  postings. 

c.  Continue  to  improve  our  Conditions  of  Service. 

d.  Maintain  a  committed,  strong  and  caring  command  chain. 

e.  Recognize  achievement  and  encourage  enterprise. 

f.  Maintain  our  commitment  to  quality  in  all  aspects  of  our  Service. 

Goal  5:  Comnunity  Relations 

a.  Encourage  cooperation  and  involvement  in  community  activities. 

b.  Provide  timely  support  to  the  civil  community  in  times  of  crisis 
and  natural  disasters. 

c.  Assist  in  search  and  rescue  operations. 

Goal  6:  International  Relations 

a.  Continue  exchange  postings  and  visits. 

b.  Contribute  to  the  Defence  Cooperation  Program  (DCP). 

c.  Participate  in  airman-to-airman  talks  with  allies  and  regional 
neighbors. 

d.  Contribute  to  United  Nations  peace-keeping  activities. 
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Appendix  D:  An  Hypothetical  Exaispla  of  the  Us*  of  the  Proposed 

Effectiveness  Metric 


To  demonstrate  the  use  of  the  proposed  effectiveness  measures,  an 
hypothetical  example  relating  to  the  Strike/Reconnaissance  PEG  was 
constructed.  Table  1  represented  the  flying  hour  program  for  week  30  of 
a  52  week  budget  period.  The  table  provided  details  of:  the  flying 
hours  scheduled  for  week  30;  the  actual  hours  flown  for  each  scheduled 
mission;  a  statement  "yes"  or  "no"  as  to  whether  the  aircraft  departed 
on  time;  and  a  statement  "yes"  or  "no"  as  to  whether  the  mission  was 
successfully  completed.  A  letter  was  placed  after  each  entry  with  a 
"no”  statement  to  indicate  the  reason  for  the  "no"  statement.  These 
notes  were  as  follows: 

(A)  The  aircraft  was  refueled  late. 

(B)  The  inertial  navigation  system  failed  before  the  mission  could  be 
completed . 

(C)  The  maintenance  by  the  Flightline  crew  was  delayed  by  the  late 
arrival  of  an  LRU,  sourced  from  the  Intermediate  Level  Maintenance 
Squadron. 

(D)  The  camera  on  the  aircraft  malfunctioned  before  the  aircrew  could 
complete  their  reconnaissance  mission. 

These  causes  for  the  non-completion  of  effective  hours  were  used 
to  attribute  lost  effective  hours  to  the  appropriate  support  component. 
The  refueling  problem  was  attributed  to  the  Flightline  for  failing  to 
coordinate  the  required  fuel  supply.  The  inertial  navigation  system 
failure  was  attributed  to  the  Logistics  Management  Squadron  because  they 
had  responsibility  for  maintaining  reliability  levels  for  the  aircraft 
and  its  systems.  The  delayed  LRU  was  attributed  to  the  Flightline, 
Supply  Support  Squadron,  Intermediate  Level  maintenance  Squadron,  and 
the  Logistics  Management  Squadron  because: 

a.  the  Flightline  was  unable  to  get  the  aircraft  to  launch  on  time; 
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TABLE  1:  HYPOTHETICAL  FLYING  PROGRAM  FOR  WEEK  30 


Flight 

Scheduled 

Hours 

On-Time 

Completed 

Number 

Hours 

Flown 

Departure 

Mission 

1 

3 

3.0 

Yes 

Yes 

2 

3 

2.9 

Yes 

Yes 

3 

4 

4.2 

Yes 

Yes 

4 

2 

1.9 

Yes 

Yes 

5 

3 

3.1 

Yes 

Yes 

6 

3 

2.8 

Yes 

Yes 

7 

4 

3.8 

No 

Yes 

8 

2 

2.0 

Yes 

Yes 

9 

4 

4.1 

Yes 

Yes 

10 

4 

4.2 

Yes 

Yes 

11 

3 

3.1 

Yes 

Yes 

12 

4 

3.8 

Yes 

Yes 

13 

3 

3.1 

Yes 

Yes 

14 

4 

4.3 

Yes 

Yes 

15 

1 

1.0 

Yes 

Yes 

16 

4 

4.2 

Yes 

Yes 

17 

2 

1.9 

Yes 

No 

18 

3 

2.9 

Yes 

Yes 

19 

2 

1.0 

Yes 

Yes 

20 

1 

1.1 

Yes 

Yes 

21 

1 

0.9 

Yes 

Yes 

22 

2 

2.2 

Yes 

Yes 

23 

2 

2.1 

Yes 

Yes 

24 

2 

1.9 

Yes 

Yes 

25 

1 

1.2 

Yes 

Yes 

26 

1 

1.1 

Yes 

Yes 

27 

2 

1.8 

Yes 

Yes 

28 

3 

3.2 

No 

Yes 

29 

2 

2.0 

Yes 

Yes 

30 

3 

3.1 

Yes 

Yes 

31 

1 

1.0 

Yes 

Yes 

32 

2 

2.2 

Yes 

Yes 

33 

1 

1.3 

Yes 

Yes 

34 

3 

2.4 

Yes 

No 

35 

3 

3.1 

Yes 

Yes 

36 

3 

2.9 

Yes 

Yes 

37 

1 

1.1 

Yes 

Yes 

38 

4 

4.3 

Yes 

Yes 

39 

3 

2.8 

Yes 

Yes 

40 

3 

2.9 

Yes 

Yes 

41 

1 

1.0 

Yes 

Yes 

42 

3 

3.2 

Yes 

Yes 

43 

3 

3.4 

Yes 

Yes 

44 

2 

2.1 

Yes 

Yes 

45 

3 

3.1 

Yes 

Yes 

Totals 

114 

114.7 

Effective 
Hours  Lost 


4  (A) 


2  (B) 


3  (C) 


3  (D) 


12 
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b.  the  Supply  Support  Squadron  was  unable  to  supply  the  LRU  requested 
by  the  Flightline  on  time; 

c.  the  Intermediate  Level  Maintenance  Squadron  failed  to  have 
sufficient  LRUs  returned  to  a  serviceable  condition  in  time  for 
the  Supply  Support  Squadron  to  issue  the  LRU  to  the  Flight line; 
and 

d.  the  Logistics  Management  Squadron  had  overall  responsibility  for 
managing  the  LRU  reparable  pipeline  and  allowed  the  pipeline  to 
fail  in  its  support  of  aircraft  operations. 

The  allocation  of  lost  effective  hours  was  summarized  in  Table  2. 
The  week  30  flying  hour  program  for  the  whole  FEG  was  then  summarized  in 
Table  3,  and  this  summary  was  transferred  to  the  year-to-date  summary  in 
Table  4. 

Three  graphs  were  derived  from  Table  4.  Figure  1  showed  the 
proportion  of  effective  hours  achieved  against  the  scheduled  flying  hour 
progrcun.  This  proportion  appeared  in  the  form  of  a  run  chart  from  which 
any  obvious  trends  could  be  determined.  Naturally,  as  with  any  process, 
this  chart  would  indicate  a  degree  of  variability  within  the  process  of 
generating  effective  hours.  Figure  2  showed  a  graph  of  the  proportion 
of  effective  hours  per  scheduled  hours  on  a  year-to-date  basis.  This 
provided  an  indication  of  the  likely  year-end  proportion  of  effective 
hours,  allowing  the  FEG  Commander  to  see  if  he  was  likely  to  meet  the 
performance  targets  that  may  have  been  set  for  him.  Finally,  Figure  3 
provided  a  comparison  of  effective  hours  against  the  actual  hours  flown, 
showing  the  shortfall  of  effective  hours  against  actual  hours  flown  — 
an  indication  of  the  quality  of  the  flying  hour  program  in  terms  of  FEG 
effectiveness  goals. 

As  the  use  of  the  effective  hours  data  was  difficult  to  adapt  to 
an  Statistical  Process  Control  (SPC)  chart,  an  SPC  chart  was  adopted  to 
monitor  the  number  of  effective  flights,  an  effective  flight  being  a 
flight  which  represented  the  achievement  of  effective  hours.  This 
allowed  the  data  to  be  treated  on  the  basis  of  the  attribute  of  the 


TABLE  2;  EFFECTIVE  HOURS  FOR  EACH  FEG  COMPONENT 


Scheduled 

Hours 

Attributed 

Losses 

Effective 

Hours 

Effective 

Percentage 

Flight line 

114 

7 

107 

94 

Supply  Support 

114 

3 

111 

97 

Intermediate 

Maintenance 

114 

3 

111 

97 

Depot  Maintenance 

114 

0 

114 

100 

Logistics  Management  114 

8 

106 

93 

TABLE  3:  WEEKLY  SUMMARY 

—  WEEK  30 

Budget  Hours 

110 

Scheduled  Hours 

114 

Hours  Flown 

114.7 

Effective  Hours  Lost 

12 

Effective  Hours  Flown 

102 
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TABLE  4:  YEAR-TO-DATE  PERFORMANCE  FOR  WHOLE  FE6 


Week 

Budgeted 

Hours 

Scheduled 

Hours 

Hours 

Flown 

Effective 

Hours 

Effective 

Percentage 

1 

60 

52 

50.7 

47 

90 

2 

fO 

60 

62.3 

60 

100 

3 

60 

63 

61.9 

58 

92 

4 

80 

82 

83.4 

77 

94 

5 

110 

114 

114.4 

103 

90 

6 

110 

106 

105.3 

101 

95 

7 

110 

110 

110.9 

98 

89 

8 

110 

111 

111.1 

104 

94 

9 

110 

115 

114.5 

102 

89 

10 

110 

92 

93.1 

81 

88 

11 

110 

112 

112.4 

99 

88 

12 

110 

112 

112.2 

103 

92 

13 

110 

108 

107.2 

101 

94 

14 

110 

109 

109.4 

97 

89 

15 

110 

113 

112.7 

98 

87 

16 

110 

108 

107.6 

101 

94 

17 

110 

110 

110.2 

108 

98 

18 

110 

111 

111.8 

103 

93 

19 

110 

103 

103.1 

87 

84 

20 

110 

107 

106.4 

93 

87 

21 

110 

115 

113.2 

106 

92 

22 

140 

145 

147.3 

137 

94 

23 

110 

102 

103.1 

87 

85 

24 

110 

109 

106.7 

101 

93 

25 

110 

112 

112.3 

106 

95 

26 

110 

116 

117.1 

106 

91 

27 

110 

110 

109.7 

100 

91 

28 

110 

108 

107.6 

96 

89 

29 

110 

107 

106.4 

94 

88 

30 

110 

114 

114.7 

102 

89 

Totals 

3150 

3136 

3138.7 

2856 

91 
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Flying  Hours 


effective  hours  being  gained  or  lost.  The  hypothetical  data  for  these 
effective  flights  were  given  in  Table  5.  The  resulting  SPC  P-chart, 
using  a  100%  variable  scunple  size  of  the  scheduled  flights,  was  provided 
in  the  graph  in  Figure  4.  The  graph  indicated  that  the  process  was  in 
control.  However,  the  FEG  Commander  could  be  concerned  with  the  degree 
of  variability  in  the  process,  and  could  start  to  identify  management 
initiatives  which  could  reduce  this  variability,  and  perhaps  improve  the 
overall  level  of  effectiveness. 


78 


TABLE  5:  DATA  ON  EFFECTIVE  FLIGHTS 


Week 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 


Scheduled 

Flights 

21 

22 

28 

33 

46 

44 

44 

44 

46 
36 

45 
48 

43 

44 

40 

43 

44 

47 

41 

43 

46 
58 
39 

44 

45 

48 

44 
39 

42 

45 


Effective 

Flights 

18 

22 

26 

31 
42 

42 

40 

41 
40 

32 
40 

43 
40 
40 
35 

40 
43 
43 
35 
37 

42 
55 

34 

41 

43 
45 
39 

35 

36 
41 


Total  1252  1141 
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Figure  4. 


P-Chart  Showing  Effective  Flights  as  a 
Proportion  of  Scheduled  Hours 


% 
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